Status Report (June 24- June 30)

1. Partially ported the current implementation

Not done:
1. Use VIGRA watershed filtering from the LazySnapping class

Next 3 weeks
-------------
6th week (July 1 - July 7)
1. Use VIGRA watershed filtering from the LazySnapping class
2. Further work on interactive image segmentation
3. add option for choosing segmentation technique

7th week (July 8 - July 14) Midterm evaluation week
1. Apply segmentation on the overlapped region
2. work on preview window (eg. show images that are selected and not selected)
3. Select multiple images for display

8th Week (July 15 - July 21)
1. determine which image is being edited
2. Start working on integration with Hugin
3. Fine-tuning mask in polygon mode

Status Report (June 17- June 23)

+ Implemented Basic polygon masking
1. Load an image: File->Load Images->(select an image)
2. Edit->Add Point (zoom in/out if requried)
3. Left click to add point and right click to finish adding points
4. File->Save Mask->(provide a filename and location) mask will be stored as a bitmap file
+ Interface between maskeditor and segmentation algorithms designed and implemented
+ Added command history support
+ Added option for saving mask bitmap
+ Fixed editing scaled image

Not done yet:
--------------
1. Finding overlapped area

Plans for next 3 weeks:
-------------------------

5th week
1. Port the current implementation
The implementation would atleast be able to work with small images
2. Use VIGRA watershed filtering from the LazySnapping class

6th week
1. Further work on interactive image segmentation
2. Apply segmentation on the overlapped region
3. work on preview window
4. Select multiple images for display

7th week
1. add option for choosing segmentation technique
2. determine which image is being edited

Status Report (June 11- June 16)

+ load mask project file and PTO
I created an ad-hoc project file called mask editor project (mep) which contains a link to the pto file and filenames of remapped images.
+ modified how project files and images get loaded
+ added zoom in/out feature (Basic features at the moment. More zooming features will be added after the core functionalities have been implemented)
+ updated design and added sequence diagram for loading mask project files
+ fixed some gui issues related to splitter window
+ Using VIGRA for doing watershed filtering
I used the watershed filtering example in VIGRA to perform watershed filtering on some of the images that I took. It took about 1 sec for 200x133 and ~20min for 3008x2000 size images. I'll be looking for alternate implementation of watershed algorithm and also try out applying the filter on the overlapped region between remapped images. I've also tried mean-shift (EDISON) and it produces an over-segmented image (3008x2000) in ~5mins.
+ Implementing polygon masking
Only the basic polygon editing mode was implemented.

Not done yet:
--------------
1. Polygon masking
This is going to be the next task. I'll be designing the generic segmentation module while working on this.

Plans for next 3 weeks:
-------------------------
The basic framework is more or less done. I'm going to focus on the core segmentation part in the next three weeks.

4th week
1. Polygon masking
2. Find common overlapped area (I'll be working with uncropped images produced by PTuncrop)
2. Start incorporating interactive segmentation
3. Refine Design related to segmentation
4. Add Undo/Redo feature (partial). Use GlobalCmdHist and Command pattern

5th week
1. work on interactive image segmentation (experiment with different energy term, pre-segmentation, color space and document the process)

6th week
1. Further work on interactive image segmentation
2. work on preview window

Status Report (June 3- June 10)

Things that have been done

1. Used HuginBase::ImageCache for loading images
2. Made some GUI changes
3. Added feature for drawing brush strokes.(polygon hasn't been added yet but it shouldn't be any different from handling brush strokes). To see the brush strokes feature -
a. Load an image
b. Hold down the left (foreground) or right (background) mouse button and drag. The stroke will be shown while dragging.

Not done yet (these will be done over the next few weeks)
1. Work on current implementation of interactive image segmentation (work with large images, and restarting flow)
2. Apply watershed filtering to the images (tentative)
3. Use hugin base especially GlobalCmdHist and Command pattern and utilities for tracing

Plan for next few weeks
3rd week
1. Read PTO File
2. Add Zoom in/out feature
3. Use VIGRA for doing basic image processing (try watershed filtering)
4. Implement basic polygon masking ie. the user will draw a polygon and the bounding region will be considered as a mask.


4th week
1. Start incorporating interactive segmentation
2. Refine Design related to segmentation
3. Add Undo/Redo feature (partial). Use GlobalCmdHist and Command pattern

5th and 6th week
1. work on interactive image segmentation (experiment with different energy term, pre-segmentation, color space and document the process)

Status Report (May 26 - June 2)

Things that have been done last week:

1. Work on current implementation of interactive image segmentation (work with large images):

The current implementation crashes when large images (eg. 3008x2000) are loaded. This happened because of using double datatype for edge weight. Using float datatype solves the problem. However large images are still an issue because of long running time. To improve performance Lazy Snapping uses watershed filtering (basically any type of filtering that preserves the edges should work) which over-segments the image and thereby effectively reducing the number of required nodes. I'm also thinking about restricting the part of the image where mask is going to be created by asking the user to draw a bounding rectangle. Although this is not a general solution but its good enough for most cases. Also if I get time I'll try doing automatic segmentation using "Efficient Graph-Based Image Segmentation" algorithm.

I applied "Efficient Graph-Based Image Segmentation" on a 3008x2000 pixel ie. 6016000 nodes. It took about 2mins to process and the resulting number of nodes was 5657. Which is very good however, lower number of component means that some disparate regions will get merged. So, I've to find a parameter values that keeps the number of nodes to something like 100000 uniform sized regions.

2. Build a bare-bone GUI with toolbars that can load images.
Done and added to svn.

3. Work on designing the class diagram
Done and zargo files are added to svn.

4. Produce a rough outline of how the mask editor can be incorporated with the preview window:

I don't have a detailed plan of this but what I'm thinking is that the PanoPreviewPanel has to be replaced with MaskEdPanel which will work as the mediator between different components required for masking.


Plan for next week:
I'll be focusing more on the framework.
1. Work on current implementation of interactive image segmentation (work with large images, and restarting flow)
2. Use HuginBase::ImageCache to load images
3. Implement the classes and refine the design if necessary. Add features like drawing brush strokes and polygon
4. Use hugin base especially GlobalCmdHist and Command pattern and utilities for tracing
5. Apply watershed filtering to the images (tentative)

Status Report (May 19 - May 25)

Things that have been done:
1. Design (Use case and Class Diagram)
2. Started porting exisiting implementation to use wxWidget and VIGRA
I've set up a basic wxWidget project. I'll start using VIGRA from the second week of coding phase.
3. Finalized scope (ie. deliverables for the project, what will be supported by the design and implementation)
Its mainly in the wiki. To summarize -
1. Create mask from brush strokes provided by the user
2. Fine tune mask
3. GUI features:
3.1 Undo/Redo
3.2 Load/Store
3.3 Zoom in/out
4. Extensible Design ie. provision for using different segmentation techniques, integrate with different front end
5. Standalone application that reads the pto file, lets the user create masks, remaps the mask and writes out the mask
6. Lastly work on automatically segmenting a sequence of images and masking out dark regions

New ideas and requirements

A lot of new ideas and requirements came out from the discussion in the mailing list. I'll summarize some of the points here for later reference.

Option for storing and loading masks:
Currently there are three options available - 1) storing the user input (this should make editing easier) 2) storing pixel-based mask 3) storing spline-based representation. There are two options for storing the mask. If its a spline representation then the splines can be stored in .pto file. In all other cases and also in general, it would be better to store the mask in a separate file and only the link will be provided in the .pto file.

At the beginning, I'll mainly focus on storing the user input and recomputing the mask from the input when its loaded.

Mask representation:
It would be nice to have the mechanism to convert pixel-based mask representation to spline-based representation and vice-versa. This is advantageous in two ways - a) a lot of tools already handle this type of representation b) spline-representation can be conveniently stored in .pto files. Autotrace tool was suggested in the discussion to compute the spline representation pixel-based mask.

This is definitely an important part. But it may not be possible implement this functionality during the summer. However the design will have provision for incorporating this feature

Fine tuning masks:
As for fine tuning masks, in general it is possible to get good results even if the mask is not completely accurate. But there are cases when good accuracy is essential.

Nona masks:
Applying masking in the nona stage may improve nona's performance. I'll be looking at this over the next few days.

Remapping masks:
It was pointed out that, when remapping is done it would be better for masks to be remapped as well. Since then users don't have to recreate the masks after remapping. To figure out how this can be done I'll be looking at PTMender.

Design:
It's important to have a loosly coupled system so that the underlying implementation is independent of the GUI.

This can probably be best represented using Command pattern. The command invoker (editor GUI) does not know how the operations are handled and the receiver (component handling masking) is not concerned about how the request was produced. To support undo/redo Memento pattern can be used. Hugin already makes use of these patterns so the implementation of these patterns in the mask editor will be kept consistent with that of Hugin. Additionally, for conversion between different formats strategy pattern can be used.