| Project: Summer of AI | |
|---|---|
  Summer of AI | |
| Naam | Summer of AI |
| Door | Jos2 |
| Status | Uitvoer |
| Madskillz | Prompt engineering, brain farts |
| Doel / Omschrijving | |
| Deze zomer, in de hackerspace, een server die lokaal LLM en image generation modellen kan draaien. | |
| Alle Projecten - Project Toevoegen | |
noezDeze zomer staat er tijdelijk een extra server in de space, deze heeft 384gb aan RAM en twee 14 core Xeon processoren. Met deze server kunnen we testen wat er allemaal lokaal kan met LLMs (vergelijkbaar met ChatGPT) en image generation (vergelijkbaar met Dall-E, maar dan veel uitgebreider).
De server is bereikbaar op de WiFi in de space op adres http://ai.hackershack.net Kan je de server niet bereiken, dan kan het zijn dat de server uit staat om energie te besparen. Zie #Aanzetten/uitzetten
Server specs
- Dell PowerEdge R630
- 2x Intel Xeon E5-2699 v4 (2,2GHz, 22C/44T, 55MB Cache, 9,6GT/s QPI, 145W, Turbo, HT), Max. 2.400MHz
- 24x 16gb = 384gb RAM
- 3x 1TB Samsung QVO SSDs
- Linux Debian 12 zonder GUI, op de bare metal hardware
Let op: Er zit geen videokaart in die gebruikt kan worden om de AI modellen op te draaien. De server gebruikt enkel de processoren, de performance is daardoor lager dan bij een server uitgerust met een videokaart.
SSH toegang
Er draait een OpenSSH server op de machine, deze is te bereiken via ssh://ai.hackershack.net:22
Er zijn twee gebruikers ingesteld:
- root
- Wachtwoord: smartblockchain
- user
- Wachtwoord: smartblockchain
Voel je vrij om nieuwe gebruikers aan te maken of je eigen public key toe te voegen aan een authorized_keys bestand. Ook extra software of nieuwe models kunnen via SSH/SFTP op de server gezet worden.
Aanzetten/uitzetten
De server zet zichzelf elke nacht om 3 uur uit. Dit is als achtervang, wanneer men is vergeten de server uit te zetten na gebruik.
Aanzetten
- Loop naar het kantoor in de kubus.
- Onder de tafel, waar alle servers op liggen te zoemen, ligt een server op een Ikea Lack tafeltje.
- Druk op het mini knopje links bovenin aan de voorkant van de server. Dit knopje moet je iets langer ingedrukt houden dan bij een laptop/PC.
- De server begint binnen de kortste keren te blazen alsof het systeem op wil stijgen. Dit is normaal, na het opstarten is de server weer lekker stil.
Uitzetten
- Login via SSH als root.
- Draai het commando poweroff.
Large Language Models (LLMs) met Ollama en OpenWebUI
Direct naar: http://ai.hackershack.net:8080
Op de server valt met enkele Large Language Models te spelen middels de OpenWebUI via de link hierboven. Deze user interface lijkt veel op de ChatGPT, Google Gemini en Claude web interfaces. Je kan een gebruiker aanmaken en dan heb je je eigen plekje voor je chats met de verschillende LLM modellen.
Op de achtergrond draait Ollama, deze schijnt ook een WebAPI te exposen.
Models en performance
Enkele voorbeeld prompts en hun performance op deze hardware.
Mistral:7b
- RAM vereist
- ~6gb
- Inlaadtijd
- enkele seconden
- Prompt
- Why is the sky blue?
- Respons tijd
- 24 seconden
- Respons lengte
- 106 woorden
- Woorden per seconde
- ~4.5 (0.22s per woord)
Respons
The sky appears blue due to a process called Rayleigh scattering. As sunlight passes through Earth's atmosphere, the smaller air molecules scatter short-wavelength light (blue and violet light) more than longer wavelengths (red, orange, and yellow). However, we see the sky as blue instead of violet because our eyes are more sensitive to blue light and because sunlight reaches us more from the blue part of the spectrum rather than the violet due to atmospheric filtering. Also, the Earth's atmosphere scatters violet light towards the sides more than it does blue light, which causes sunsets and sunrises to often appear red or orange instead of violet.
DeepSeek V3:671b
- RAM vereist
- ~420gb
- Inlaadtijd
- ~15 minuten
- Prompt
- Why is the sky blue?
- Respons tijd
- 44 minuten 20 seconden
- Respons lengte
- 206 woorden
- Woorden per seconde
- ~0.07 (~13s per woord)
Respons
The sky appears blue due to a phenomenon called Rayleigh scattering. Here's how it works:
- Sunlight and Atmosphere: Sunlight, which appears white, is made up of different colors (wavelengths) of light. When sunlight enters Earth’s atmosphere, it interacts with the gases and particles in the air.
- Scattering of Light: Shorter wavelengths of light—like blue and violet—are scattered more effectively by the molecules in the atmosphere than longer wavelengths like red or yellow. This scattering occurs because the size of these atmospheric molecules is much smaller than the wavelength of visible light.
- Why Blue, Not Violet?: Although both blue and violet are scattered more, our eyes are less sensitive to violet light and more sensitive to blue. Additionally, some of the violet light is absorbed by the upper atmosphere.
- Result: When we look up at the sky during the day, we see the scattered blue light coming from all directions, making the sky appear blue.
At sunrise or sunset, the sky appears red or orange because the sunlight has to travel through more of the atmosphere. This causes most of the shorter wavelengths (blue and violet) to be scattered out of view, leaving the longer wavelengths (red and orange) to dominate.
DeepSeek R1:671b
- RAM vereist
- ~420gb
- Inlaadtijd
- ~15 minuten
- Prompt
- Why is the sky blue?
- Respons tijd
- 52 minuten 33 seconden
- Respons lengte
- 619 woorden
- Woorden per seconde
- ~0.2 (~5s per woord)
Respons
Thinking...
Okay, the user asked "Why is the sky blue?" This seems like a simple question but it's actually a great example of basic physics. I remember this involves Rayleigh scattering.
First, I should start with sunlight being white light made up of all colors. Then explain how when that light enters Earth's atmosphere, it interacts with gas molecules—mainly nitrogen and oxygen. The key point is that these molecules are much smaller than the wavelength of visible light.
I need to emphasize that shorter wavelengths (blue/violet) scatter more efficiently because scattering intensity goes as 1/λ⁴. But wait... if violet scatters most strongly, why isn't the sky purple? Ah yes—because our eyes are less sensitive to violet and there's some atmospheric absorption too. Also sunlight has slightly more blue than violet.
Hmm should I mention how this differs at sunrise/sunset when light travels through more atmosphere? That might help explain why we see reds then. Oh and maybe clarify that other colors like green/yellow do scatter, just less intensely—so they contribute to the overall blue by blending with it?
The user seems curious but probably not looking for super technical details. Keeping explanations vivid would help—like comparing sky color to "blue light" hitting our eyes from all directions. Should avoid jargon unless absolutely necessary.
Also wondering... is this a child asking? The simplicity suggests maybe yes. But could also be an adult refreshing basic knowledge. Either way, using concrete examples (sunlight through prism) makes it accessible. Ending with that sunset connection might satisfy deeper curiosity about related phenomena.
...done thinking.
The sky appears blue due to Rayleigh scattering, which describes how sunlight interacts with Earth's atmosphere:
- Sunlight Composition: Sunlight is "white light" made up of all visible colors (red, orange, yellow, green, blue, indigo, violet), each with a different wavelength.
- Interaction with Atmosphere: As sunlight enters the atmosphere, it collides with gas molecules (mostly nitrogen and oxygen) and tiny particles.
- Selective Scattering:
- Gas molecules are much smaller than the wavelengths of visible light.
- Shorter wavelengths (blue/violet end of the spectrum) scatter more efficiently than longer wavelengths (red/orange/yellow).
- The intensity of scattering is inversely proportional to the fourth power of the wavelength (I ∝ 1/λ⁴). This means blue light (~450 nm) scatters about 9-10 times more strongly than red light (~650 nm).
- The Blue Sky Effect:
- The scattered blue light is sent in all directions by countless molecules across the sky.
- When you look up (away from the direct sun), this scattered blue light reaches your eyes from many different points, making the entire dome of the sky appear blue.
Why isn't it Violet?
Violet has an even shorter wavelength and scatters more than blue. However:
- Sunlight contains slightly less violet to begin with.
- Our human eyes are much less sensitive to violet light compared to blue light.
- Some higher-altitude atmospheric absorption occurs for violet wavelengths.
As a result, the predominant color we perceive from scattered sunlight is blue.
Related Effects:
- Red Sunrises/Sunsets: When the sun is low on the horizon, its light passes through much more atmosphere. Most of the shorter-wavelength blues and greens are scattered away long before reaching your eyes. The longer wavelengths (reds, oranges) dominate the direct sunlight you see.
- White Horizon Sky Near Sun: Looking near a sunset or sunrise sky close to the sun, light has passed through so much air that even some blue gets rescattered back towards your line of sight, mixing with other colors and appearing whitish.
So, it's not that the atmosphere emits blue light; rather, its molecules selectively scatter incoming sunlight's shorter wavelengths (blue/violet) in all directions, overwhelming our eyes with blue.
Image Generation met ComfyUI
Direct naar: http://ai.hackershack.net:8188
Met de ComfyUI web interface kan je je eigen image generation workflow opzetten en draaien. Denk aan het koppelen van allerlei blokken om zo een ketting van verschillende bewerkingen op te zetten. Uiteindelijk komt er een plaatje uit en deze is in de gallery voor iedereen in de space te zien, en te verwijderen.
In de gallery kan de gebruikte workflow van een gegenereerd plaatje gekopieerd worden, deze kan je in een nieuwe workflow in ComfyUI plakken en aanpassen of opnieuw draaien.
Let op: De PNGs die ComfyUI genereerd hebben standaard de workflow in de IHDR staan. Dit is een plain text JSON string die in de image file staat. In deze string staan bijvoorbeeld de gebruikte blokken, verbindingen, model namen, text prompts en bestandsnamen (bij image-to-image generation). Wil je de gegenereerde image gebruiken, maar niet de workflow meesturen? Strip dan alle metadata uit de PNG, bijvoorbeeld met ImageMagick.
Models en performance
Enkele voorbeelden van de modellen.
DreamShaper XL
"DreamShaper is a general purpose SD model that aims at doing everything well, photos, art, anime, manga. It's designed to go against other general purpose models and pipelines like Midjourney and DALL-E."
- Recommended settings
- KSampler cfg
- 2
- KSampler steps
- 4 - 8
- KSampler sampler_name
- DPM++ SDE
- KSampler scheduler
- Karras (or Normal)
DreamShaper AbsoluteReality
"Long story short, DS aims at art, this aims at realism."
- Recommended settings
- KSampler cfg
- 4.5 - 10
- KSampler steps
- 25 - 30
- KSampler sampler_name
- DPM++ SDE
- KSampler scheduler
- Karras
- Negative prompt
- BadDream, (UnrealisticDream:1.2)
DreamShaper 8
"I think the "swiss knife" type of model is closer then ever. The primary goal [of this model] is still towards art and illustrations. Being good at everything comes second."
- Recommended settings
- KSampler cfg
- ~2
- KSampler steps
- 5 - 15
- KSampler sampler_name
- LCM
Stable Diffusion 1.5
"Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input."
- Recommended settings
- Unknown
Stable Diffusion XL Turbo 1.0
"SDXL-Turbo is a distilled version of SDXL 1.0, trained for real-time synthesis."
- Recommended settings
- KSampler steps
- 1 - 4
Flux 1 [dev]
"FLUX.1 [dev] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions."
- Recommended settings
- KSampler cfg scale
- 3.5
- KSampler steps
- 50
Flux 1 [schnell]
"Trained using latent adversarial diffusion distillation, FLUX.1 [schnell] can generate high-quality images in only 1 to 4 steps."
- Recommended settings
- KSampler cfg scale
- 0
- KSampler steps
- 1 - 4