Crumb

DanQuix0te

u/danQuix0te@feed.timeloop.tv
26 posts · 31 comments

Recent posts

Recent comments

on AI-WebTV Prompt Guide · c/txt2vid · 1 pts · 3y

Database here - https://huggingface.co/spaces/jbilcke-hf/media-server/raw/main/database.json

Example -

{
  "version": 1,
  "startAtShotId": "",
  "sequences": [
    {
      "sequenceId": "d344637d-247f-4dd1-8700-02741f36a94a",
      "skip": false,
      "lastGenerationAt": "",
      "videoPrompt": "3D animation, blender renderings, simulations showcasing a variety of different effects",
      "audioPrompt": "electronic goal music from the 90s",
      "tags": [
        "trailer",
        "cgi"
      ],
      "channel": "main",
      "shots": [
        {
          "shotId": "fa3b74d1-f27f-4fc4-a5f3-f29dd0d2750e",
          "index": 0,
          "lastGenerationAt": "",
          "videoPrompt": "stunning 3D video of rippling water waves, reflected on hovering metal beads in an underwater setting. Rendered in Blender with subtle, whaling ambient light, captured in 8K UHD.",
          "audioPrompt": ""
        },
        {
          "shotId": "f89e600c-91e8-401d-a870-3286b779e5a3",
          "index": 1,
          "lastGenerationAt": "",
          "videoPrompt": "high-quality 3D animation of a cloth-like structure falling in a windless room, displaying the realistic physics of fabric movement. Rendered in ultra-detailed Blender, shot in 4K Cinema4D.",
          "audioPrompt": ""
        },
        {
          "shotId": "333aae1c-db8c-424e-bb05-d7161952beea",
          "index": 2,
          "lastGenerationAt": "",
          "videoPrompt": "splendid video showing the fluid simulation of liquids as they interact with static and moving objects, set in a minimalistic, clean environment. Captured with 4K Cinema4D, showcases Blender render.",
          "audioPrompt": ""
        },
        {
          "shotId": "a19a3e22-9009-449c-a011-6d6cdc660477",
          "index": 3,
          "lastGenerationAt": "",
          "videoPrompt": "detailed 3D animation of cellular automata, using simple elements to create complex patterns in a uniform space. Exhibits award-winning lighting techniques and Blender rendering, shot in Cinema4D studio quality.",
          "audioPrompt": ""
        },
        {
          "shotId": "694cb3a9-7f51-4f7e-86df-7ae56d34e433",
          "index": 4,
          "lastGenerationAt": "",
          "videoPrompt": "fabulous Blender rendering of a high-speed camera capture of metal beads impacting a flat surface, displaying brilliant n-body collision physics. Captured at a ripe evening with Cinema4D.",
          "audioPrompt": ""
        },
        {
          "shotId": "d59ec76d-4769-44ab-9aea-09362764d35d",
          "index": 5,
          "lastGenerationAt": "",
          "videoPrompt": "breathtakingly crisp 3D animation featuring a fluid mechanism, where treacle-thick liquid pours into a large vat. Beautifully lit by diffused, soft light. Rendered in Blender, captured at 8K UHD with Cinema4D.",
          "audioPrompt": ""
        },
        {
          "shotId": "e170953f-4913-428d-a551-53128920e4c8",
          "index": 6,
          "lastGenerationAt": "",
          "videoPrompt": "surreal yet hyper-realistic 3D animation of cloth simulation, showcasing the realism of Blender's material simulation capabilities. With studio lighting illuminating the scene, shot at close-up with Cinema4D.",
          "audioPrompt": ""
        },
        {
          "shotId": "1dcca91d-c9a4-4ed2-b011-1a55d990ca46",
          "index": 7,
          "lastGenerationAt": "",
          "videoPrompt": "contemporary fine-art 3D animation showcasing the stunning interaction between thousands of metal beads under n-body simulation. Rendered with precision in Blender and captured at a high detail with Cinema4D.",
          "audioPrompt": ""
        },
        {
          "shotId": "a9296116-b5e5-43ef-914e-d75c7ad2750e",
          "index": 8,
          "lastGenerationAt": "",
          "videoPrompt": "Blender-rendered 3D animation of a glass cube falling into a pool of liquid, captured to showcase fluid dynamics and refraction of light effects. Caught under a soft diffused light with Cinema4D.",
          "audioPrompt": ""
        },
        {
          "shotId": "6c8e7622-268e-491a-a6bc-0ad5b927c7c0",
          "index": 9,
          "lastGenerationAt": "",
          "videoPrompt": "impressive 3D animation, of cellular automata, forming fascinating patterns in a confined environment. The geometrically exact elements are rendered with Blender. Shot at dusk, in crisp 8K UHD with Cinema4D.",
          "audioPrompt": ""
        }
      ]
    },
{
on AI WebTV - jbilcke-hf · c/txt2vid · 1 pts · 3y

Some notes from here: https://huggingface.co/spaces/jbilcke-hf/AI-WebTV/discussions/8

The main code of the webtv is located inside the media-server :

manual steps:

  • human input to write a short paragraph describing a multi-shot video sequence
  • manual submit it to GPT-4 to generate a list of video captions for each shot (the system instructions are extracts from a stable diffusion guide)
  • commit the captions to the playlist database

Inside the media-server space (generation process running in the background):

  • for each prompt in the database
  • generate a silent 3 seconds video clip with Zeroscope V2 576w (hosted on Hugging Face Spaces)
  • upscale the clip with Zeroscope V2 XL (also a HF Space)
  • perform frame interpolation with FILM (also a HF Space)
  • storage in the Persistent Storage of the media-server Space

Inside the media-server space (streaming process running in the foreground):

  • for each video file in the persistent storage folder
  • add it to a new FFmpeg playlist (it's just a .txt file)
  • broadcast it over the RTMP protocol using FFmpeg (in FLV format)
  • diffusion of the stream using node-media-server

Inside the AI-WebTV space:

  • display the stream using mpegts.js
  • this doesn't work on iPhone, but now there is also a Twitch mirror
on federation test · c/selfhosted · 0 pts · 3y

A new instance doesn't federate on its own. It needs to discover the other servers. Use something like https://lemmyverse.net/ to make links with your home server and visit the other instances. Subscribe to a few and you'll start getting those posts.

on Space I - ZeroScope v2 · c/txt2vid · 1 pts · 3y

Adding some quality descriptors seemed to help.

 {
'prompt': 'Traveling through space, stars, galaxies, rainbow of colors, cinematic, imax, movie, film grain', 
'n_prompt': 'text, watermark, copyright, blurry, nsfw', 
'steps': 30, 
'frames': 24, 
'seed': 3824253423, 
'scale': 17, 
'width': 576, 
'height': 320, 
'eta': 0.0, 
'cpu_vae': 'GPU', 
'device': device(type='cuda'), 
'skip_steps': 0, 
'strength': 0
}

on Space I - ZeroScope v2 · c/txt2vid · 1 pts · 3y

Another generation with the same prompt at ZeroScope 576w default resolution of 576 x 320.

{
'prompt': 'Traveling through space, stars, galaxies, rainbow of colors', 
'n_prompt': 'text, watermark, copyright, blurry, nsfw', 
'steps': 30, 
'frames': 24, 
'seed': 4037714052, 
'scale': 17, 
'width': 576, 
'height': 320, 
'eta': 0.0, 
'cpu_vae': 'GPU', 
'device': device(type='cuda'), 
'skip_steps': 0, 
'strength': 0
}

on federation test · c/selfhosted · 1 pts · 3y

No worries! All tests welcome in this thread. Do you think it's your server host? What's the output of the "date" command?