Feedback

[jmacey]

This is a really good project idea. I would suggest starting on a small subset of rot areas first (i.e. not hair!)

For the cryptomatte side of things you will need some form of api to write this data out. I think some of it can be found here https://github.com/Psyop/Cryptomatte not sure if there is an official pip one. You may also need to install OpenEXR in python to be able to write out the matte to exr https://pypi.org/project/OpenEXR/

I would try the methods of installation here https://github.com/facebookresearch/sam2 first and see if they work in the labs. if there are issues let me know and we can figure it out.

[jmacey]

Have you had any more ideas / progress on this project? Hopefully some of the things we have done in the labs this week will give you more ideas etc.

we will start to look at setting up a project in the labs this week this will be a good place to start and you can begin to add notebooks etc to the project so I can see in more detail what you are doing and how to approach things.

[jmacey]

try to start setting up some of the basic notebooks and DataLoaders you may need over the next few weeks to try and get the project started.

[jmacey]

the notebooks seem to be coming on well and there is some good basic progress. I will look into the cryptomatte more. I have made a little more progress so hopefully will have something to write them next week.

[jmacey]

Good progress, I've not had a chance to look at the cryptomatte stuff yet will try after xmas. Do you have any worked examples of using the output yet? It is always good to try and get the final product setup to give the full pipeline a test (and evaluate the processes).

[alexmoed]

Hi Jon,
Thanks for looking over my files. I'll make sure to be more diligent with checking and documenting my progress moving forward. I was able to perform testing and training using the images in my dataset, and it works ( some images work really nicely, others less so, haha). I also loaded a video, created masks, and exported it as an EXR sequence with masks stored in separate channels. I'll document everything here right after Christmas.
Sounds good about the cryptomattes. Thanks again for working on it!

Merry Christmas!

[alexmoed]

Hi Jon.

The readme and my files have been updated.

Cheers.
Alex

[alexmoed]

Hi Jon,
Quick follow-up—I’ve prepared the files for the lab. The script that processes a video and generates the EXRs works fine on our workstations. The training script is also prepared for the lab, but since the GPU there has less memory than Google Colab, I get better results on Colab and cant run it at the same settings. Would it be okay if I did the demo video for the training on Google Colab and then created a second video for the video-to-roto EXR process on the lab computers?
Cheers.
Alex

[jmacey]

Sure that should be fine. Have you tried it on the teaching machine as this may have a bit more GPU ram than your machine.

[alexmoed]

Thanks! I haven't tried that yet. I'll give it a shot tomorrow. Also, quick question: I remembered it mentioned, but can you remind me what you are looking for in the dataset ethics checklist?

[jmacey]

Just a general check to make sure the data is open and not restricted etc. (Ideally all rights have been paid etc if you can find it!)