Das Haus - Barbie

I was recruited to do visuals for a private Halloween party. The theme of the party was "Nuclear Barbie" - My goal was to use projection mapping to turn the house into a creepy post-apocalyptic dollhouse. My plan was to use ControlNet with a sketch of the house to generate visuals that worked with the house architecture.

To accomplish this, I first went to the house and mocked out where I'd set up the projector, and used MadMapper to project a grid on the house to outline where I'd want the projection to be. We didn't want projections going in the window, so I wanted to create content that would be somewhat aware of the windows so the visuals could move naturally around them. I took a photo of the grid, and then using Gimp cropped it and stretched it so that the grid was roughly square.

Here's the original photo:

And here's the stretched, cropped, and aligned photo:

From there, I used Gimp to draw the outline of the house, adding extra details to help guide ControlNet towards the type of dollhouse I had in mind.

Then, I fed the sketch and prompt (something like, "the outside of a pink dollhouse filled with radioactive sludge and mutant monsters") into ControlNet and let it generate some starting images.

Here's some of the starting images:

Then, I just did img->img from the starting images to generate a video using Stable Diffusion. I projected those videos on the house for the party. Here's how it turned out: