Description
«Green Memory» is an experiment focused on training the generative neural network Stable Diffusion XL using original nature photography.
The project was based on a collection of 35 photographs taken during environmental observations. The shots feature grasses, flowers, trees, bodies of water, park spaces, and natural details.
The goal of the experiment is to investigate how capable a generative model is of absorbing the visual characteristics of a personal photo archive and applying them to create new images.
Unlike standard text-prompt generation, this process involves training the neural network additionally on a specific set of images. This allows for an exploration of how individual visual material influences the model’s output.
Dataset
A dataset of 35 original nature photographs, each sized 1024×1024 pixels, was compiled for the training.
The collection unites various natural subjects: flowering plants, grassy areas, trees, woodland patches, bodies of water, and urban vegetation.
Particular attention in the photographs is given to natural colors, the diversity of plant forms, the textures of leaves and flowers, and light.
The dataset includes not only natural landscapes but also images of plants in an urban environment. This allows the model to work with different types of compositions—from individual flowers to more complex spaces.
All images in the dataset are original photographs and were used for training in compliance with the assignment requirements.
Image examples from the dataset
Model Training
The generative model Stable Diffusion XL 1.0 was used for this project.
Training was conducted using the DreamBooth LoRA method.
The Kaggle platform, equipped with an NVIDIA Tesla T4 GPU, served as the computational environment.
Learning Parameters:— Base Model: Stable Diffusion XL 1.0 — Method: DreamBooth LoRA — Dataset Size: 35 photographs — Original Resolution: 1024 × 1024 pixels — Training Resolution: 512 × 512 pixels — Number of Steps: 500 — Batch Size: 1 — Gradient Accumulation: 4 — Learning Rate: 0.0001 — Precision: FP16 — Optimizer: 8-bit AdamW — Training Seed: 42
Textual descriptions were automatically generated for each image using the BLIP model. These were augmented with a general formulation in the SVETA style, which served as a learned style identifier.
After the data preparation, LoRA training was conducted. This resulted in the file pytorch_lora_weights.safetensors, which contains the trained parameters.
To generate new images, LoRA was connected to the base model, Stable Diffusion XL.
Result
The project resulted in a series of six images dedicated to various forms of natural space. A common stylistic identifier, «nature in SVETA style,» was used for all generations. The subjects were defined by separate text prompts.
- prompt = «nature in SVETA style, wild meadow with flowers, soft morning light, detailed plants, natural landscape»
- prompt = «nature in SVETA style, close-up of delicate wildflowers growing among green grass, soft natural sunlight, detailed petals and leaves, realistic nature photography»
- prompt = «nature in SVETA style, lush green forest with tall trees, dense vegetation, soft sunlight filtering through leaves, peaceful natural atmosphere, realistic nature photography»
- prompt = «nature in SVETA style, peaceful small pond surrounded by lush green grass and wildflowers, reflections of trees in still water, soft natural daylight, realistic nature photography»
- prompt = «nature in SVETA style, a narrow, winding path through a lush green meadow overgrown with tall grass and delicate wildflowers, soft sunlight, peaceful atmosphere, realistic nature photography»
- prompt = «nature in SVETA style, a close-up of delicate green leaves, wild grasses, and tiny plants, sunlight shining through translucent leaves, intricate natural textures, soft background, realistic macro nature photography»
Stable Diffusion XL, trained with a LoRA, was used to generate the images.All prompts included the phrase «nature in SVETA style.» Only the scenes were varied: meadows, flowers, forests, ponds, and other natural elements.Generation parameters: 30 steps, guidance scale 7.5, LoRA weight—0.7.No additional image processing was applied.
Comparison
Looking at the first two images, one can observe the preservation of several characteristic features of the original dataset. First and foremost is the prevalence of natural green hues. Vegetation occupies nearly the entire composition, creating a dense, continuous sense of space. The second important element is the diversity of plant forms. The images feature numerous stems, leaves, grasses, and small flowers that form a multi-layered structure. The third characteristic is the photographic nature of the images. The generations aim to reproduce natural lighting, spatial depth, and the heterogeneity of the vegetation.
The original photographs show bright garden flowers, dense vegetation, and elements of the urban environment.The generated images feature more uniform plants: they are dominated by green grasses, small flowers, and soft, sunny lighting. The neural network retained the natural theme but created new compositions.Differences between the generations: The first image presents a wide shot of a blooming meadow. The flowers are small and evenly distributed among the grass.In the second image, the flowers are closer to the viewer. White petals and yellow centers stand out against the green background.Both pieces share a similar color palette but differ in scale and composition.
Conclusions
Through training on 35 original photographs, a LoRA was created to generate natural imagery. The model allowed for the creation of a series of six works united by a botanical theme and a similar color palette. However, the images vary in composition and scale. The project demonstrates how personal photographs can be utilized to build a new visual series using neural networks.
