01110111 01110100 01100110

Ctrl + C / Ctrl + v

Ongoing experiments

Experiments run on eden.art
Links to AI models used are included below

This is a set of AI-driven projects investigating one of AI’s most prominent features, style transfer, in custom applications. Two avenues of experimentation are:

1. Exploring design space that is contextually relevant to the project(Since the images are generated with existing images from the project, further generations are similar enough that they are ~feasible~ as compared to only using text-prompting / image-prompting.)

2. Recreating representational style that matches the existing data (Since the model is built on existing imagery, no style edits are required to make these images presentation ready. The only downside is that generations are always raster images and not easily editable vectors)

experiment #1

A close friend and colleague, Daniel Wong, was interested in the lifespan of buildings and their components for his thesis work, and in the process had drawn enough images that a clear visual style was established. Using these drawing sets we trained two LORA’s and generated new images.

Try it out yourself here.

dataset - drawings by Daniel Wong

AI outputs

At this time I was also working on my own thesis regarding trees, so a lot of the input images are from my research material. These had very interesting outcomes, as a lot of the outputs are very consistent on composition and color. The model also skews heavily in favor of isometric and 2-point perspective views, which matches the views used in the training data.

The odd generation is actually this image below, the input is a plate from a set of section/elevations of how ship masts are composited. The input dataset however contains no drawings of that view. I’m guessing that this output is a result of some influence from the main AI model and not from the LORA, or from some of the generic word prompted included like “architecture, decay, rubble, rebuild” etc.

more technical explanation: Ok so a huge misnomer is that people think of “AI” and “AI models” as a singular entity that takes some kind of input and gives some kind of output. AI models are more like a cake that has multiple layers, each layer processes the input image into other forms of information (vectors, words etc.) and at the end outputs another image. The LORA (Low-Ranking Adaptation) here is a layer that heavily influences and guides the output image into a particular style, and are relatively easy to train as compared to the main AI model, which requires a far larger dataset and more computational power. You can also stack LORA’s to really refine your output results, think of them as “data add-ons” that help you generate images your main AI model isn’t well-versed in.

experiment #2

This dataset is twice as large as the previous, and is very consistent in visual style and framing. While the style outputs consistently, I wasn’t able to recreate the frame around images.

Try this model out here

dataset - drawings by Daniel Wong

AI outputs

The results from this dataset were oddly more varied than I had anticipated, especially considering the consistency of the training data. More experiments need to be run altering the “concept strength” to higher values. While overall graphic style is consistent, I wasn’t able to get the image frame from the training data to appear, but this might be due to aspect ratio differences and/or a lack of white space in the input imagery. Also, I’m finding myself drawn to using colorful input imagery since the correlation between input/output comes through very clearly. Product photography tends to be of high quality and clear so it has been a good starting point for image generation.

Note to self: use a taxonomy diagram to view all image inputs, outputs and parameters.

experiment #3

Here I’m training an AI dataset using my own thesis drawings, testing to see what kind of “new” log variations can possibly be made, and mixing it with different text/image prompts. While these are interesting generations, they are typically not feasible to fabricate.

Try this model out

dataset