11th Week (Aug 5 - Aug 11)

1. Further work on Load/Store mask
It is now possible to store the bitmap mask while saving the project. However I haven't yet figure out a good way to pass the edited alpha channel to the stitchers used by the preview frame or use the transformation library.
2. Bug fix:
a) fixed crash while changing mask editing mode
b) fixed editing problem while zoomed in/out
3. Wiki updated.

Not done yet:
==========
1. Work on remapping
2. Pre-segmentation in lazy snapping
3. Lazy snapping polygon boundary editing. Implement contour finding.

12th Week (Aug 12 - Aug 18) [Aug 18th is final day for uploading code]
1. Allow editing the vertices of the polygon in the polygon editing mode
2. Store the mask after proper scaling
3. Preliminary implementation for pre-segmentation and contour finding
4. Update design documents

10th Week (July 29 - Aug 4)

1st half of 10th Week July 29 - July 31
1. Implemented Undo/Redo feature for basic polygon editing (only available in standalone maskeditor)
2. Implement image loading by using remapped images from PreviewPanel
However the edited alpha layer needs to be fed from the MaskEdEditWnd

2nd half of 10th Week (July 29 - Aug 4)
1. Improve usability. Determine which image is being edited, polygon editing
Implemented in integrated version. Basically if a single image is chosen then that image is considered as being edited otherwise the last image of multiple selection is considered the top image
2. Fixed mask editing bug. The polyed_basic can be used to edit mask. However there's another bug that prevents changing mask editing mode. This will be fixed in the next commit.
3. Further work on integration. Use edited alpha layer.
There's still some work to do here. Basically converting from wxBitmap to BImage and using that mask.

Not done yet:
==============
1. Load/Store mask
This hasn't been implemented yet. I'm thinking about doing an inverse mapping of the mask created from the preview mode and incorporating it with the original file.


11th Week (Aug 5 - Aug 11) [Aug 11 is suggested pencils down]
1. Further work on Load/Store mask
2. Work on remapping
3. Usability: work on zooming
4. Pre-segmentation in lazy snapping
5. Lazy snapping polygon boundary editing. Implement contour finding.

12th Week (Aug 12 - Aug 18) [Aug 18th is final day for uploading code]
1. Usability
2. Integration

9th Week (July 22 - July 28)

1. MaskEdEditWnd integration with PreviewFrame
It is now possible to switch to Mask Editor mode. However image loading needs to be modified. I'm thinking of using the UpdatePreview implemented in PreviewPanel to create individual remapped image and then feeding that to MaskEdEditWnd as well as MaskMgr.

31st July
1. Undo/Redo feature for basic polygon editing
2. Implement image loading as describe above
3. Incorporate pre-segmentation step in lazy snapping
4. Load/Store mask

10th Week (July 29 - Aug 4)
1. Further work on Load/Store mask
2. Implement basic mechanism for doing remapping
3. Improve usability. Determine which image is being edited, polygon editing
4. Further work on integration
5. Lazy snapping polygon boundary editing.


11th Week (Aug 5 - Aug 11) [Aug 11 is suggested pencils down]
1. Further work on remapping
2. Usability: work on zooming
3. Integration

12th Week (Aug 12 - Aug 18) [Aug 18th is final day for uploading code]
1. Usability
2. Integration

8th Week (July 15 - July 21)

1. Work on fixing lazysnapping
Lazy Snapping is working now however it needs some improvement in terms of speed and performance.
2. Started working on integration with Hugin.
a)merged from trunk to branch.
b)Changed build system. Now the maskeditor core and segmentation routines are built as libraries which can be used by either the standalone gui or hugin.
c)Started modifying PreviewFrame to incorporate MaskEdEditWnd

Not done yet:
=============
1. Determine which image is being edited

9th Week (July 22 - July 28)
1. Undo/Redo feature
2. Load/Store mask
3. Further work on integration. In particular allow basic polygon editing.
4. Incorporate pre-segmentation step in lazy snapping.

10th Week (July 29 - Aug 4)
1. Further work on Load/Store mask
2. Implement basic mechanism for doing remapping
3. Improve usability. Determine which image is being edited, polygon editing
4. Further work on integration
5. Lazy snapping polygon boundary editing.


11th Week (Aug 5 - Aug 11) [Aug 11 is suggested pencils down]
1. Further work on remapping
2. Usability: work on zooming
3. Integration

12th Week (Aug 12 - Aug 18) [Aug 18th is final day for uploading code]
1. Usability
2. Integration

Status Report (July 8 - July 14)

7th week
1. Used crop information to determine overlapped rectangular region
2. Preview window now allows (de)selecting images.
however editing option isn't updated yet.
3. Fixed clustering problem in LazySnapping. However there're still some other bugs.

Plan for next 3 weeks:
================
8th Week (July 15 - July 21)
1. Determine which image is being edited
2. Start working on integration with Hugin. In particular allow switching between preview mode and mask editing mode
3. Work on fixing lazysnapping

9th Week (July 22 - July 28)
1. Undo/Redo feature
2. Load/Store mask
3. Further work on integration (I'll determine the deliverables on 8th week)
4. Further work on lazysnapping. If its working properly then incorporate pre-segmentation step.

10th Week (July 29 - Aug 4)
1. Implement basic mechanism for doing remapping
2. Improve usability
3. Further work on integration

Status Report (July 1 - July 7)

6th week (July 1 - July 7)
1. Implemented LazySnapping without watershed filtering. The implementation doesn't quite work yet.
At the moment I'm using a naive implementation of K-means. But I'm thinking of using the implementation provided at http://www.cs.umd.edu/~mount/Projects/KMeans/ (release under GPL)
2. add option for choosing segmentation technique
Initially basic polygon editing is chosen. This can be changed from Edit->Preferences. The segmentation technique can be changed anytime. However the previous masks (if any) are discarded. Later on I'll provide option for reusing previous mask.

Not done yet
--------------
1. Working implementation of LazySnapping.
2. Did not make use of watershed filtering. I'm thinking of just using watershed.cxx code which is part of VIGRA examples. The only thing I'll need to do is build a graph from the output of the watershed algorithm.


Plan for second phase:
---------------------------
For the second phase I'm planning to give high priority to integration and usability (eg. fast response, easily going back and forth between different images that are edited) and slightly lower priority to fine-tuning, and specialized mask creation. Plan for the next three weeks is as follows -


7th week (July 8 - July 14) Midterm evaluation week
1. Apply segmentation on the overlapped region
2. work on preview window (eg. show images that are selected and not selected)
This is basically giving a preview of loaded images. I'll keep it very simple for now.
3. Select multiple images for display
4. Work on LazySnapping. I'll mainly be focusing on the part without watershed filtering.
5. Determine which image is being edited

8th Week (July 15 - July 21)

2. Start working on integration with Hugin
3. Start working on fine-tuning mask in polygon mode
4. Do further improvement of lazysnapping.

9th Week (July 22 - July 28)
1. Undo/Redo feature
2. Load/Store mask
3. Further work on integration (I'll determine the deliverables on 8th week)

Status Report (June 24- June 30)

1. Partially ported the current implementation

Not done:
1. Use VIGRA watershed filtering from the LazySnapping class

Next 3 weeks
-------------
6th week (July 1 - July 7)
1. Use VIGRA watershed filtering from the LazySnapping class
2. Further work on interactive image segmentation
3. add option for choosing segmentation technique

7th week (July 8 - July 14) Midterm evaluation week
1. Apply segmentation on the overlapped region
2. work on preview window (eg. show images that are selected and not selected)
3. Select multiple images for display

8th Week (July 15 - July 21)
1. determine which image is being edited
2. Start working on integration with Hugin
3. Fine-tuning mask in polygon mode

Status Report (June 17- June 23)

+ Implemented Basic polygon masking
1. Load an image: File->Load Images->(select an image)
2. Edit->Add Point (zoom in/out if requried)
3. Left click to add point and right click to finish adding points
4. File->Save Mask->(provide a filename and location) mask will be stored as a bitmap file
+ Interface between maskeditor and segmentation algorithms designed and implemented
+ Added command history support
+ Added option for saving mask bitmap
+ Fixed editing scaled image

Not done yet:
--------------
1. Finding overlapped area

Plans for next 3 weeks:
-------------------------

5th week
1. Port the current implementation
The implementation would atleast be able to work with small images
2. Use VIGRA watershed filtering from the LazySnapping class

6th week
1. Further work on interactive image segmentation
2. Apply segmentation on the overlapped region
3. work on preview window
4. Select multiple images for display

7th week
1. add option for choosing segmentation technique
2. determine which image is being edited

Status Report (June 11- June 16)

+ load mask project file and PTO
I created an ad-hoc project file called mask editor project (mep) which contains a link to the pto file and filenames of remapped images.
+ modified how project files and images get loaded
+ added zoom in/out feature (Basic features at the moment. More zooming features will be added after the core functionalities have been implemented)
+ updated design and added sequence diagram for loading mask project files
+ fixed some gui issues related to splitter window
+ Using VIGRA for doing watershed filtering
I used the watershed filtering example in VIGRA to perform watershed filtering on some of the images that I took. It took about 1 sec for 200x133 and ~20min for 3008x2000 size images. I'll be looking for alternate implementation of watershed algorithm and also try out applying the filter on the overlapped region between remapped images. I've also tried mean-shift (EDISON) and it produces an over-segmented image (3008x2000) in ~5mins.
+ Implementing polygon masking
Only the basic polygon editing mode was implemented.

Not done yet:
--------------
1. Polygon masking
This is going to be the next task. I'll be designing the generic segmentation module while working on this.

Plans for next 3 weeks:
-------------------------
The basic framework is more or less done. I'm going to focus on the core segmentation part in the next three weeks.

4th week
1. Polygon masking
2. Find common overlapped area (I'll be working with uncropped images produced by PTuncrop)
2. Start incorporating interactive segmentation
3. Refine Design related to segmentation
4. Add Undo/Redo feature (partial). Use GlobalCmdHist and Command pattern

5th week
1. work on interactive image segmentation (experiment with different energy term, pre-segmentation, color space and document the process)

6th week
1. Further work on interactive image segmentation
2. work on preview window

Status Report (June 3- June 10)

Things that have been done

1. Used HuginBase::ImageCache for loading images
2. Made some GUI changes
3. Added feature for drawing brush strokes.(polygon hasn't been added yet but it shouldn't be any different from handling brush strokes). To see the brush strokes feature -
a. Load an image
b. Hold down the left (foreground) or right (background) mouse button and drag. The stroke will be shown while dragging.

Not done yet (these will be done over the next few weeks)
1. Work on current implementation of interactive image segmentation (work with large images, and restarting flow)
2. Apply watershed filtering to the images (tentative)
3. Use hugin base especially GlobalCmdHist and Command pattern and utilities for tracing

Plan for next few weeks
3rd week
1. Read PTO File
2. Add Zoom in/out feature
3. Use VIGRA for doing basic image processing (try watershed filtering)
4. Implement basic polygon masking ie. the user will draw a polygon and the bounding region will be considered as a mask.


4th week
1. Start incorporating interactive segmentation
2. Refine Design related to segmentation
3. Add Undo/Redo feature (partial). Use GlobalCmdHist and Command pattern

5th and 6th week
1. work on interactive image segmentation (experiment with different energy term, pre-segmentation, color space and document the process)

Status Report (May 26 - June 2)

Things that have been done last week:

1. Work on current implementation of interactive image segmentation (work with large images):

The current implementation crashes when large images (eg. 3008x2000) are loaded. This happened because of using double datatype for edge weight. Using float datatype solves the problem. However large images are still an issue because of long running time. To improve performance Lazy Snapping uses watershed filtering (basically any type of filtering that preserves the edges should work) which over-segments the image and thereby effectively reducing the number of required nodes. I'm also thinking about restricting the part of the image where mask is going to be created by asking the user to draw a bounding rectangle. Although this is not a general solution but its good enough for most cases. Also if I get time I'll try doing automatic segmentation using "Efficient Graph-Based Image Segmentation" algorithm.

I applied "Efficient Graph-Based Image Segmentation" on a 3008x2000 pixel ie. 6016000 nodes. It took about 2mins to process and the resulting number of nodes was 5657. Which is very good however, lower number of component means that some disparate regions will get merged. So, I've to find a parameter values that keeps the number of nodes to something like 100000 uniform sized regions.

2. Build a bare-bone GUI with toolbars that can load images.
Done and added to svn.

3. Work on designing the class diagram
Done and zargo files are added to svn.

4. Produce a rough outline of how the mask editor can be incorporated with the preview window:

I don't have a detailed plan of this but what I'm thinking is that the PanoPreviewPanel has to be replaced with MaskEdPanel which will work as the mediator between different components required for masking.


Plan for next week:
I'll be focusing more on the framework.
1. Work on current implementation of interactive image segmentation (work with large images, and restarting flow)
2. Use HuginBase::ImageCache to load images
3. Implement the classes and refine the design if necessary. Add features like drawing brush strokes and polygon
4. Use hugin base especially GlobalCmdHist and Command pattern and utilities for tracing
5. Apply watershed filtering to the images (tentative)

Status Report (May 19 - May 25)

Things that have been done:
1. Design (Use case and Class Diagram)
2. Started porting exisiting implementation to use wxWidget and VIGRA
I've set up a basic wxWidget project. I'll start using VIGRA from the second week of coding phase.
3. Finalized scope (ie. deliverables for the project, what will be supported by the design and implementation)
Its mainly in the wiki. To summarize -
1. Create mask from brush strokes provided by the user
2. Fine tune mask
3. GUI features:
3.1 Undo/Redo
3.2 Load/Store
3.3 Zoom in/out
4. Extensible Design ie. provision for using different segmentation techniques, integrate with different front end
5. Standalone application that reads the pto file, lets the user create masks, remaps the mask and writes out the mask
6. Lastly work on automatically segmenting a sequence of images and masking out dark regions

New ideas and requirements

A lot of new ideas and requirements came out from the discussion in the mailing list. I'll summarize some of the points here for later reference.

Option for storing and loading masks:
Currently there are three options available - 1) storing the user input (this should make editing easier) 2) storing pixel-based mask 3) storing spline-based representation. There are two options for storing the mask. If its a spline representation then the splines can be stored in .pto file. In all other cases and also in general, it would be better to store the mask in a separate file and only the link will be provided in the .pto file.

At the beginning, I'll mainly focus on storing the user input and recomputing the mask from the input when its loaded.

Mask representation:
It would be nice to have the mechanism to convert pixel-based mask representation to spline-based representation and vice-versa. This is advantageous in two ways - a) a lot of tools already handle this type of representation b) spline-representation can be conveniently stored in .pto files. Autotrace tool was suggested in the discussion to compute the spline representation pixel-based mask.

This is definitely an important part. But it may not be possible implement this functionality during the summer. However the design will have provision for incorporating this feature

Fine tuning masks:
As for fine tuning masks, in general it is possible to get good results even if the mask is not completely accurate. But there are cases when good accuracy is essential.

Nona masks:
Applying masking in the nona stage may improve nona's performance. I'll be looking at this over the next few days.

Remapping masks:
It was pointed out that, when remapping is done it would be better for masks to be remapped as well. Since then users don't have to recreate the masks after remapping. To figure out how this can be done I'll be looking at PTMender.

Design:
It's important to have a loosly coupled system so that the underlying implementation is independent of the GUI.

This can probably be best represented using Command pattern. The command invoker (editor GUI) does not know how the operations are handled and the receiver (component handling masking) is not concerned about how the request was produced. To support undo/redo Memento pattern can be used. Hugin already makes use of these patterns so the implementation of these patterns in the mask editor will be kept consistent with that of Hugin. Additionally, for conversion between different formats strategy pattern can be used.

Masking in GUI

In this summer as a participant of Google Summer of Code 2008, I'll working on a GUI based blend mask editor for hugin/panotools community under the guidance of Daniel M. German. This blog is mainly to keep track of my work and also to update interested people about the project. I'll also maintain a wiki for documenting the project. Please take a look at it and the references therein to get an idea of what the project is about and how its going to be done.