×
Namespaces

Variants
Actions

Creating an edition history with undo and redo actions optimized for Nokia imaging SDK

From Nokia Developer Wiki
Jump to: navigation, search

This article shows how work an edition history and how to implement the most appropriate edition history according to the app functionality.

Note.pngNote: This is an entry in the Nokia Imaging and Big UI Wiki Competition 2013Q4.

SignpostIcon XAML 40.png
WP Metro Icon WP8.png
Article Metadata
Code ExampleTested with
Devices(s): Nokia Lumia 520
Compatibility
Platform(s):
Windows Phone 8
Dependencies: Nokia Imaging SDK 1.0
Article
Created: juan_K (15 Dec 2013)
Last edited: chintandave_er (20 Dec 2013)


Contents

Nokia imaging SDK 1.0

With the arrival of Nokia imaging sdk, the doors opened to a new world of possibilities on the photo editing and capture development for Windows phone. So far Nokia imaging sdk was still in beta version, but last November, Nokia finally released the Nokia Imaging SDK version 1.0 This release brought significant changes and improvements over the beta versions (also some bumps). If you used a beta version of the SDK take a look to this article, is a good base to upgrade to version 1.0. If you're new to the SDK is recommended to review this quick start. You can download the Nokia Imaging SDK via Nuget here.

Introduction

An important point that must be taken into account when an application that allows the user to make changes to any file (including images, text or other file type) is developed, is to add a "edition history" that allows the user to undo changes, and / or redo. The previous version of the SDK included an Undo into the EditingSession within class (responsible for all processing of the image), allowing undo the last action performed on the image.

With the arrival of the new version of the SDK, the EditingSession class was removed, and replaced by the "edition pipeline", although it has a very practical and versatile operativity, does not provide a direct method to undo changes (or redo). However, the sample application from Nokia FilterExplorer have a custom implementation of the undo method, this implementation appears appears to have the same functionality as the Undo method of the old EditingSession class.

In this tutorial, we will see in general mode what is an edition history, undo and redo actions, edition history implementation and optimization according to the specific functionality that have your application.


Edition History

Both big desktop applications (office suite, Photoshop, pait ! , Etc) , and mobile applications where the primary ( or secondary) functionality is based on the edition, is good for the user the possibility of have control over the changes being made , undo bad changes and / or redo if desired. However, it is not always necessary for an application to use an edition history , or use one very complex. For example instagram allows the user to apply one and only one filter to a photo, add a frame if desired and blur then he can share their photography, there is no real need to go back a step .

Other applications require only undo the action , without using the redo action into the edition history . That is why I decided to show how to implement an edit history , which is specifically optimized to fit different requirements.

Usually, this is the basic pattern that follows a normally edition history:

Pattern.png

In the graph, the figures with the A are the object we are editing ( image in our case) , and objects with the P are the edition processing (filters) we performed. Basically , A1 passes through the processing P1 resulting on the object A2 and so on until A4, the Undo action allows the user to back to the A3 object, and in turn causes the existence of the Redo action, where the user can go again to the A4 object. We can see that the session history could just use the undo action , but never could use just the redo action, because this is only generated when the user undo any changes.

Being clear about this basic pattern of edition, we can think of several different implementations of the edition history , then is necessary to know what Edition history gives us better performance, quality and flexibility, taking into account the functionality of our application.


Different edition history for different needs

While I was developing a photo editing app for Windows Phone Store, I had to try different ways to implement an edit history, each had its advantages and disadvantages over the other, making them useful for different types of application.

These are the types of edit history that I categorize, based on the pattern shown above

1. Processing based edition history:

This type of history is used by the application FilterExplorer. Here, instead of making a processing of mode A1 -> P1 -> A2 -> P2 -> A3 ..., is used a list, where each processed (sdk filter) is stored and then is performed a single edition step where all filters included in the list are aplied, can be represented as: A1 -> NP -> AX, where PN is the complete list of processed and AX is the final result. Thus, you can go directly from A1 to A4 as shown below:

Process based pattern.png

To implement undo the action, simply delete the last item on the list of processed, and execute the complete edition process again. If you want to implement further action remake, simply add an auxiliary list, which will store the waste filters instead of deleting them. If you've seen before FilterExplorer the app, you'll know that this is the method used to undo the filters applied. For more information abour FilterExplorer, go here The application can be downloaded here if you want to check the code directly.

This type of history has the initial advantage of producing little memory consumption, but has a problen when a big number of filters are added, the first drawback of this type of history is that while more filters are added to the list, the processing becomes slower , this makes it inconvenient if you want to develop an app that allows you to do many filters or generate previews. Another drawback of this kind of history used with the Nokia sdk , is precisely that required to keep all filters , remember that there are filters in the sdk that inherit from IDisposable or use Idisposables resources , some of them are curves, blend and imagefussion. Thus, this type of history is recommended , only if your application does not allow to generate real-time previews , avoids the use of filters to use or inherit from IDisposable. In the specific case of FilterExplorer application , we can see that the application maintains an initial decent performance, but but to apply many filters become slow.

2. Object based edition history:

 This type of history , instead of keeping a list of filters to be applied , use a list of the objects that passing through the editing process, is fully adjustable to the pattern: A1 - > P1 - > A2 -> P2 - > A3..., apply a processing while keeping saved on a list each resulting object , for implement the undo action just get the last item in the list, and make a rendering. Also if we include a function to redo just move the last item on the list to other list before apply the undo action. The main advantage of this method is is that the processing speed is stable throughout the running of the application, also we can use Idisposable filters or filters that use Idisposables resources , is also very useful if you want to have real-time previews (such as Instagram ) . Now a great drawback is that store one by one the objects are editing generates excessive memory consumption, while processing functions maintain a good speed, the total ram of the app will go to the clouds once a large number of filters is applied. In a basic implementation of this type of edition history , is recommended that the app is described as high memory consumption, would be useful keep an eye on memory consumption to inform the user before an overflow occurs.

A very useful way to avoid excessive memory is usage is use temporary objects, save and load them on the phone as needed to undo or redo. When the editing process are complete, or just when the app is closed,the temporary folder and the temporary files are deleted. Applying this technique, we can get very good results not only in the performance of image processing, but on the overall application performance.

3. Edition history based on restoration of processing:

The history based on restoration processing , requires that objects processing (filters in our case) are stored, following the normal pattern edition A1 - > P1 -> A2 -> P2 - > A3 ... here , when you undo an action, instead of removing the last filter and apply it all again , we "invest " the processing , for example, if we apply a filter that makes the alpha channel of the image converts to 0, which would invest this for returns to be 255 or otherwise , the default value of the channel . This kind of history is perhaps the most complex and complete , and that promises better performance with fewer disadvantages , however it is currently not possible to directly use it in conjunction witthh the pipeline and the default filters of the SDK. For example, if we apply a brightness filter to an image with the 0.2 value , reapply the filter with the inverted value don't help. What seems feasible is to use this method to create custom filters and code this to make possible the invest.

Note.pngNote: At the time i'm not test this history, but with the theoretical explanation , you can get an idea of ​​how to implement at least some custom filters.

4. Custom History:

Basically is to use a portion of each of the first two types of history . Processed are stored in a list, but will only use it when the Undo action is required , and we will have a single object that will serve as a source for processing and will also become the final result. The editing pattern would remain similar to the history of object-based editing , with the difference that all objects are not stored. If the user wants to go a step back then eliminate the last object in the list of processed and all were applied again. Although this type of history is somewhat promising , it also has some drawbacks. The problem of the disposable objects is remains, the undo actions would be really slow when stacking many filters in the list , however allows previews in real time with good performance and does not cause excessive memory usage . There is a major concern with this type of history , and has to do with the outcome of the images and previous undo action using views. It happens that each record generates final results of the different image , thus if the user uses the undo action , it may not be a 100% equal to the last picture. This topic will be discussed later.

If you do not have problems to the undo action is somewhat slow, and using filters where the application mode variation is not very visible (using filters such as brightness, contrast, etc..'s Not very visible change when using the undo) this method is indeed a very good choice. But remember the problem of disposable objects.


Code and examples

Now that we have seen how work each edition history, and his advantages and disadvantages. Let's see some explanatory code on them. In this case I have prepared 2 different apps for Processing based and Object based history because put this together in the same app make complicate get real performance information. ( also I have no more than a low memory device to make the test) . Custom history not cover in this tutorial by simply being a combination of the other two types of history .

ProcessingBasedHistory

This is the first app where we use the processing based on history , it is a simple app that allows the user to open or take a photo , then add a series of filters on another page ( much like FilterExplorer ) . PhotoManager is our core class, we will implement here our edit history , methods of processing , rendering, and further methods to save. This is our PhotoManager class:


#region Fields
 
private List<FilterModel> _filters = new List<FilterModel>();
private List<IFilter> _components = new List<IFilter>();
private List<FilterModel> _auxFilters = new List<FilterModel>();
private IBuffer _originalBuffer;
 
#endregion
 
#region Properties
 
 
public List<FilterModel> Filters
{
get
{
return _filters;
}
}
 
public bool CanUndo
{
get
{
return _filters.Count > 0;
}
}
 
public bool CanRedo
{
get
{
return _auxFilters.Count > 0;
}
}
 
public int Width { get; private set; }
 
public int Height { get; private set; }
 
public bool Dirty { get; set; }
 
#endregion
 
public PhotoManager(IBuffer buffer)
{
_originalBuffer = buffer;
Dirty = false;
}
 
public async Task LoadImageInfo()
{
await UpdateImageSize();
}
 
public void Dispose()
{
_originalBuffer = null;
_filters = null;
_components = null;
_originalBuffer = null;
}
 
#region Render Methods
 
public async Task<Bitmap> RenderThumbnailBitmapAsync(int side)
{
int minSide = (int)Math.Min(Width, Height);
 
Windows.Foundation.Rect rect = new Windows.Foundation.Rect()
{
Width = minSide,
Height = minSide,
X = (Width - minSide) / 2,
Y = (Height - minSide) / 2,
};
 
CropFilter cropFilter = new CropFilter(rect);
 
Bitmap bitmap = new Bitmap(new Windows.Foundation.Size(side, side), ColorMode.Ayuv4444);
 
using (BufferImageSource source = new BufferImageSource(_originalBuffer))
using (FilterEffect effect = new FilterEffect(source) { Filters = new IFilter[] { cropFilter } })
using (BitmapRenderer renderer = new BitmapRenderer(effect, bitmap, OutputOption.Stretch))
{
await renderer.RenderAsync();
}
return bitmap;
}
 
public async Task RenderAsync(WriteableBitmap bitmap)
{
using (BufferImageSource source = new BufferImageSource(_originalBuffer))
using (FilterEffect effect = new FilterEffect(source) { Filters = _components })
using (WriteableBitmapRenderer renderer = new WriteableBitmapRenderer(effect, bitmap))
{
await renderer.RenderAsync();
bitmap.Invalidate();
}
}
 
public void ApplyFilter(FilterModel filter)
{
Filters.Add(filter);
 
foreach (IFilter f in filter.Components)
{
_components.Add(f);
}
if (_auxFilters.Count != 0)
{
_auxFilters.Clear();
}
}
 
#endregion
 
#region History Filter Methods
 
public void Undo()
{
if (CanUndo)
{
FilterModel filter = Filters[Filters.Count - 1];
if (_auxFilters.Count == 5)
{
_auxFilters.RemoveAt(0);
}
_auxFilters.Add(filter);
for (int i = 0; i < filter.Components.Count; i++)
{
_components.RemoveAt(_components.Count - 1);
}
Filters.RemoveAt(Filters.Count - 1);
}
}
 
public void Redo()
{
if (CanRedo)
{
FilterModel filter = _auxFilters[_auxFilters.Count - 1];
Filters.Add(filter);
foreach (IFilter f in filter.Components)
{
_components.Add(f);
}
_auxFilters.RemoveAt(_auxFilters.Count - 1);
}
}
 
public void UndoAll()
{
if (CanUndo)
{
Filters.Clear();
_auxFilters.Clear();
_components.Clear();
}
}
 
#endregion
 
public Size DefineTargetScaledSize(FrameworkElement ContainerSize)
{
double currentScale;
if (Width < Height)
{
currentScale = ContainerSize.ActualHeight / Height;
}
else
{
currentScale = ContainerSize.ActualWidth / Width;
}
 
Size targetSize = new Size(Width * currentScale, Height * currentScale);
return targetSize;
}
 
#region Save Methods
 
public async Task SavePhoto()
{
string fileName = DateTime.UtcNow.Ticks.ToString();
using (MediaLibrary mediaLibrary = new MediaLibrary())
{
using (BufferImageSource source = new BufferImageSource(_originalBuffer))
using (JpegRenderer renderer = new JpegRenderer(source))
using (MediaLibrary library = new MediaLibrary())
{
IBuffer buffer = await renderer.RenderAsync();
using (Picture picture = library.SavePicture(fileName, buffer.AsStream()))
{
}
}
}
}
 
#endregion
 
#region Private Methods
 
private async Task UpdateImageSize()
{
using (BufferImageSource source = new BufferImageSource(_originalBuffer))
{
ImageProviderInfo info = await source.GetInfoAsync();
Width = (int)info.ImageSize.Width;
Height = (int)info.ImageSize.Height;
}
 
}
 
#endregion

In addition to the methods of rendering and processing, we can also see other methods to obtain the relative sizes that must be rendered, and other helper methods. We can see that _auxRedoFilters _applyedFilters and lists, are responsible for storing our processed. As you can see the basic operation of the class is to add a filter to the list _applyedFilters and then make a rendering image with all filters.

If you want to put a limit on actions that can be redone, adds an integer variable called _redoLimit to class and it defines the limit you want and change the code of Undo method for this:

public void Undo()
{
if (CanUndo)
{
FilterModel filter = Filters[Filters.Count - 1];
if (_auxFilters.Count == _redoLimit)
{
_auxFilters.RemoveAt(0);
}
_auxFilters.Add(filter);
for (int i = 0; i < filter.Components.Count; i++)
{
_components.RemoveAt(_components.Count - 1);
}
Filters.RemoveAt(Filters.Count - 1);
}
}

ObjectBasedHistory

This is my favorite history type, here implement object-based history. Unlike the first application, some will exploit the advantages of this edit history, allowing for previews of filters on the same page before applying (instagram style). This is the PhotoManager.cs class for this application:

#region Fields
 
private const string TEMPSTART = "tpe";
private const string TEMPEXTENTION = ".tmx";
private IBuffer _originalBuffer;
private IBuffer _currentBuffer;
private int _filtersCount;
private int _historyCount;
 
#endregion
 
#region Properties
 
public bool CanUndo
{
get
{
return _filtersCount > 0;
}
}
 
public bool CanRedo
{
get
{
return _historyCount > _filtersCount;
}
}
 
public double CurrentInvertedScale { get; private set; }
 
public double CurrentScale { get; private set; }
 
public int Width { get; private set; }
 
public int Height { get; private set; }
 
public bool Dirty { get; set; }
 
#endregion
 
public PhotoManager(IBuffer buffer)
{
_originalBuffer = buffer;
_currentBuffer = buffer;
Dirty = false;
_historyCount = _filtersCount = 0;
}
 
public async Task LoadImageInfo()
{
await UpdateImageSize();
await ClearTempObjects();
}
 
public void Dispose()
{
_originalBuffer = null;
_currentBuffer = null;
ClearTempObjects();
}
 
#region Temp Objects Management
 
private async Task WriteTempObject()
{
byte[] fileBytes = _currentBuffer.ToArray();
StorageFolder local = Windows.Storage.ApplicationData.Current.LocalFolder;
var dataFolder = await local.CreateFolderAsync("TempEdition",
CreationCollisionOption.OpenIfExists);
var file = await dataFolder.CreateFileAsync(TEMPSTART + _filtersCount.ToString() + TEMPEXTENTION,
CreationCollisionOption.ReplaceExisting);
IReadOnlyList<StorageFile> x = await dataFolder.GetFilesAsync();
using (var s = await file.OpenStreamForWriteAsync())
{
s.Write(fileBytes, 0, fileBytes.Length);
}
}
 
private async Task<IBuffer> LoadTempObject(int index)
{
StorageFolder local = Windows.Storage.ApplicationData.Current.LocalFolder;
IBuffer buffer;
if (local != null)
{
var dataFolder = await local.GetFolderAsync("TempEdition");
var file = await dataFolder.OpenStreamForReadAsync(TEMPSTART + index + TEMPEXTENTION);
using (StreamReader streamReader = new StreamReader(file))
using (MemoryStream stream = new MemoryStream())
{
streamReader.BaseStream.CopyTo(stream);
buffer = stream.GetWindowsRuntimeBuffer();
}
return buffer;
}
else
{
return null;
}
}
 
public async Task ClearTempObjects()
{
StorageFolder local = Windows.Storage.ApplicationData.Current.LocalFolder;
if (local != null)
{
try
{
var dataFolder = await local.GetFolderAsync("TempEdition");
await dataFolder.DeleteAsync();
}
catch (Exception e)
{
 
}
var dataFolderx = await local.CreateFolderAsync("TempEdition",
CreationCollisionOption.OpenIfExists);
}
}
 
#endregion
 
 
 
#region Render Methods
 
public async Task<Bitmap> RenderThumbnailBitmapAsync(int side)
{
int minSide = (int)Math.Min(Width, Height);
 
Windows.Foundation.Rect rect = new Windows.Foundation.Rect()
{
Width = minSide,
Height = minSide,
X = (Width - minSide) / 2,
Y = (Height - minSide) / 2,
};
 
CropFilter cropFilter = new CropFilter(rect);
 
Bitmap bitmap = new Bitmap(new Windows.Foundation.Size(side, side), ColorMode.Ayuv4444);
 
using (BufferImageSource source = new BufferImageSource(_originalBuffer))
using (FilterEffect effect = new FilterEffect(source) { Filters = new IFilter[] { cropFilter } })
using (BitmapRenderer renderer = new BitmapRenderer(effect, bitmap, OutputOption.Stretch))
{
await renderer.RenderAsync();
 
}
return bitmap;
}
 
public async Task RenderPreview(WriteableBitmap bitmap, FilterModel filter)
{
WriteableBitmap tempBitmap = new WriteableBitmap(bitmap);
using (BufferImageSource source = new BufferImageSource(_currentBuffer))
using (FilterEffect filterProcessor = new FilterEffect(source) { Filters = filter.Components })
using (WriteableBitmapRenderer renderer = new WriteableBitmapRenderer(filterProcessor, tempBitmap))
{
await renderer.RenderAsync();
tempBitmap.Pixels.CopyTo(bitmap.Pixels, 0);
}
}
 
public async Task RenderAsync(WriteableBitmap bitmap)
{
WriteableBitmap tempBitmap = new WriteableBitmap(bitmap);
using (BufferImageSource source = new BufferImageSource(_currentBuffer))
using (WriteableBitmapRenderer renderer = new WriteableBitmapRenderer(source, tempBitmap))
{
await renderer.RenderAsync();
tempBitmap.Pixels.CopyTo(bitmap.Pixels, 0);
}
}
 
public async Task RenderOriginalAsync(WriteableBitmap bitmap)
{
WriteableBitmap tempBitmap = new WriteableBitmap(bitmap);
using (BufferImageSource source = new BufferImageSource(_originalBuffer))
using (WriteableBitmapRenderer renderer = new WriteableBitmapRenderer(source, tempBitmap))
{
await renderer.RenderAsync();
tempBitmap.Pixels.CopyTo(bitmap.Pixels, 0);
}
}
 
public async Task ApplyFilter(WriteableBitmap bitmap, FilterModel filter)
{
IBuffer buffer;
using (BufferImageSource source = new BufferImageSource(_currentBuffer))
using (FilterEffect effect = new FilterEffect(source) { Filters = filter.Components })
using (JpegRenderer bufferRenderer = new JpegRenderer(effect))
using (WriteableBitmapRenderer renderer = new WriteableBitmapRenderer(effect, bitmap))
{
buffer = await bufferRenderer.RenderAsync();
await renderer.RenderAsync();
bitmap.Invalidate();
}
_currentBuffer = buffer;
_filtersCount++;
_historyCount = _filtersCount;
await WriteTempObject();
}
 
#endregion
 
#region History Filter Methods
 
public async Task Undo()
{
if (CanUndo)
{
if (_filtersCount > 1)
{
_currentBuffer = await LoadTempObject(_filtersCount - 1);
}
else
{
_currentBuffer = _originalBuffer;
}
_filtersCount--;
}
}
 
public async Task Redo()
{
if (CanRedo)
{
_currentBuffer = await LoadTempObject(_filtersCount + 1);
_filtersCount++;
}
}
 
public async Task UndoAll()
{
if (CanUndo)
{
await ClearTempObjects();
_historyCount = _filtersCount = 0;
_currentBuffer = _originalBuffer;
}
}
 
#endregion
 
public Size DefineTargetScaledSize(FrameworkElement ContainerSize)
{
double currentScale;
if (Width < Height)
{
currentScale = ContainerSize.ActualHeight / Height;
}
else
{
currentScale = ContainerSize.ActualWidth / Width;
}
 
Size targetSize = new Size(Width * currentScale, Height * currentScale);
return targetSize;
}
 
#region Save Methods
 
public async Task SavePhoto()
{
string fileName = DateTime.UtcNow.Ticks.ToString();
using (MediaLibrary mediaLibrary = new MediaLibrary())
{
using (BufferImageSource source = new BufferImageSource(_currentBuffer))
using (JpegRenderer renderer = new JpegRenderer(source))
using (MediaLibrary library = new MediaLibrary())
{
IBuffer buffer = await renderer.RenderAsync();
using (Picture picture = library.SavePicture(fileName, buffer.AsStream()))
{
}
}
}
}
 
#endregion
 
#region Private Methods
 
private async Task UpdateImageSize()
{
using (BufferImageSource source = new BufferImageSource(_originalBuffer))
{
ImageProviderInfo info = await source.GetInfoAsync();
Width = (int)info.ImageSize.Width;
Height = (int)info.ImageSize.Height;
}
 
}
 
#endregion

We can see clear differences between this class and PhotoManager class the previous application , here we do not use listed as support for nuestroas actions undo and redo, here are replaced by temporary objects stored in the phone , and counters : _filtersCount and _historyCount . Here we can see that every time you call the method applied , the WriteTempObject method is invoked, to undo an action method undo makes use of LoadTempObject method , allowing you to read the last stored object whose name matches the value of _filtersCount counter - 1 . EL method does the same redo the undo method , but load the object that matches the value of _filtersCount + 1 token . The objects were removed if Undoall method use is , if you call the Dispose method of the class , or failing to use the LoadImageInfo method, which is always called just after creating a new instance of the class PhotoManager .


Different backgrounds - different end results.

As I mentioned earlier, the final outcome of history based on objects and processed based on history can be different, it may vary depending on the filters that apply and order thereof. That is, the result of applying a filter at a time on a bufferSource and apply them all at once, it is not always the same. This is why I prefer not to use the custom history. However, it is something that is not always noticeable to the user and varies widely depending on the values ​​and order of the filters applied. I could After several attempts, making these two images look something different, although it is somewhat minimal in this case:

Edited Image

Original Image

In both images are applied the same filters, arrows and red circles show where changes in more detail. While these changes are minimal in this example, other combinations of filters can generate much more noticeable changes. The image on the left corresponds to the object based history and right to the processing based history. You can see the history and process-based in this case, some areas show the resulting blues strongest than when using the object-based history.

Final notes

After seeing how to implement different types of editing histories, their advantages and disadvantages, and the final results that each generates each developer's decision to opt for one or the other, always taking into account the functionality you want to give your application . Personally I prefer the object-based history (what I'm using in an application of photo editing I intend to climb to the store before the end of the year), but depending on the functionality of the application, the processing-based history or custom can be more efficient.

Note that applications are not optimized to be uploaded to the store, and there are still many improvements that can be applied to this item. For example, applying custom filters (you can easily add in the object-based history), compression of temporary objects, implementing the mvvm pattern, creating a dual model to store temporary objects on the phone and keep them in memory by most appropriate mode according to the features of the phone. etc. I hope this tutorial can be useful for the community as a basis for implementing their own records of publication. Any questions, recommendations or corrections are welcome, I know there are many things that need to improve, I welcome your comments.

Note.pngNote: please excuse the errors of English language to have the tutorial. I'm good at reading English, but not so good at writing it.

This page was last modified on 20 December 2013, at 10:41.
93 page views in the last 30 days.

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×