PASTEDOWN   238   0
   5181 36.54 KB    888

Preservation Project History - 2020 to 2021

By Synthbot
Created: 2021-07-16 21:31:33
Updated: 2021-06-15 01:48:51
Expiry: Never

Animation

  • Age3rcm, Blender Anon, initial proposal for how to do animation, suggestion to use derpibooru for animation data, collecting leaked animation files, learning to transfer 250+ GB to each other, Clipper sorting the leaks, just render stuff, swf decompiler, check out JSFL, realizing differences between 3D animation and 2D animation, using modern JavaScript with JSFL, learning how Animate’s APIs work, idea to have animation AI follow the same structure as animation software, AI tells us animation AI is impossible, labeling symbols, Hacker Anon teaches us to mod Adobe Animate, Render Anon, bugs in Adobe Animate

More anons joining in

  • Age3rcm, arfafax, Astralite, Render Anon, BFDIAnon (+note of how PPP positively affected his life), /g/ anons asking us for coding help, more datasets from persona nerd and gothic anon, Poopsikins

Improving TTS or using it in new ways

  • 48khz upsampling, Flutterbitch, lot of experimentation, HifiGAN, SFX and BGM, Celestia rap, singing, abridged series, comment from 15 about PPP data being the best emotive dataset, using punctuation to control speech, the cocaine saga (skits with different TTS engines talking to each other), CPU speech generation, radio show (Dash and Twi in the morning, Great and Powerful Radio Show), using audio file formats to find good sound representations, getting Festival Multisyn working, 15 adding arpabet support, CPU synthesis tool from Delta, RD Let’s Play, 15 adding torchmoji, Cookie and 15 working together, Celestia responds to Anon’s letters, War of the Worlds, beginning of a full Rarity episode, RTX voice for data cleaning, making characters whisper, Pony Thread Simulator, getting chrysalis’ vocal effects, guided meditation with Luna, the model pronounces ponepasta properly, pondering a better phonetic language than arpabet and ipa, 15 serves over 1 million clips in a single day, pitch shifting for S1 Luna, audio super-resolution

Branching out

  • Comic transcription, story labeling tool, animation, TPDNE, Blender Anon, SFX and BGM, labeling booru image tags, playing with GPT-3, creating music loops for anons playing with singing voices, first attempts at generating images, voice changer, drawing of twilight sparkle relaxing by the fireplace, greentext ai anon, pony butt dataset, StyleCLIP, Twi Chatbot from Astralite Cookie goes into debt and fucking dies Sells his home to pay for GPUS, dies of appendicitis, constantly coughing and wheezing, Cookie going into debt for the PPP, can’t believe cookie is fucking dead again

Drama & news

  • Search “discord”, “eschelon”, AI dungeon going shit, synthbot destroys cookie-15 relationship by withholding Wheatley data, random anon gets pissy about names, kids on the internet taking credit for Delta’s/Cookie’s TTS, Wubcake butthurt, vocodes is pathetic, CLIP has pony neurons, iCartoonFace dataset for research contains pony

Jun 13, the panel begins

Rewriting the Master Doc Age3rcm joins the thread:

We have no idea what we’re doing with animation AI

  • https://desuarchive.org/mlp/thread/35437062/#q35448983
  • The goal at this point was to label show animations. Maybe we’ll get to that at some point, but we have more basic tasks to do right now.
  • We hadn’t considered using the animation source files yet. We didn’t know if we could use those.

Jun 14, Age3rcm explains the hard parts of animation. With the data we’re collecting, we should be able to support all of it.

The anon also offers a puppet for us to study. We have no idea what to do with it. Anon wondering how much of episode creation we can do with AI:

Suggestion to use OpenToonz:

Some Astralite anon has been working on finetuning GPT- 2 for MLP fics.

Persona nerd dumps a bunch of data. He and GothicAnon are the two non- regulars contributing data:

Jun 15, SnoopyAnon gets the ngroks to rap:

Age3rcm suggests having a library of poses.

Reading through OpenToonz docs to understand how animation works:

Cookie training WaveFlow:

Anon transcribes the first comic with the hope of eventually getting pony chatbots:

Jun 16, Anon combines 22khz waveglow with 48khz tacotron:

Delta tries MelGAN- STFT without success:

No luck with Cookie’s WaveFlow:

Jun 18, using AI to speed up comic transcription:

Cookie’s Separable Channel WaveFlow (SmallWaveFlow)

Pony Life released We’re not clipping Pony Life.

Jun 19, Bipolar Flutterbitch

Jun 21, Delta tries out a FastSpeech 1.5, which was incorrectly labeled FastSpeech 2

Cookie fucking dies from appendicitis

Cookie tries every model that has the word “Fast” or “Flow” in it. (Plus AlignTTS)

Anon figures out how to convert 48khz spectograms to 22khz to use to fully- trained vocoder.

Jun 22, Clipper clips SFX and music.

Jun 23, Cookie gets GlowTTS working.

Jun 25, turns our we can use Google’s research TPUs for ponies

Jun 26, I start trying to clean up our data pipeline (again)

Some ideas on what we can do with Animation data:

Jun 27, Blender Anon suggests using Blender for 2D animation

Jun 27, anon suggests using JSFL to dump assets

It was more difficult than expected, but this is ultimately the approach we’re following to dump asset samples so we can label them. Anon finds a FastSpeech2 implementation that uses arpabet

Delta gets it running: - https://desuarchive.org/mlp/thread/35514662/#q35517729

Anons learning to do AI

Jesus fucking christ, it’s a dinosaur

Jun 30, Celestia rap:

Jul 1, anon discovers that Cookie’s models can sing

“tell Equestria to send a message to /mlp/con”

Jul 2, more errors in the dataset

Derpy being cute

Jul 7, Blender Anon suggests converting FLA to XFL, which is easier to work with

We’re (finally) working on this part Clipper finishes clipping SFX and BGM for the first two seasons

We start collecting all the leaked animation files:

Jul 8, initial proposal for how to do animation AI

This gets trashed when we later realize we have enough leaked files that we don’t need to reverse engineer the puppet movements. Jul 9, Cozy says the N word

Failed suggestion to use Derpibooru for animation data

Jul 10, drama over 15

We decide to spend some time labeling *booru tags that describe animations

Labeling tool: - https://desuarchive.org/mlp/thread/35582876/#q35596892

Done on Jul 18 - - https://desuarchive.org/mlp/thread/35582876/#q35628309

Jul 12, List of which characters can sing:

Jul 14, GAN Ponies (TPDNE)

Jul 19, someone makes a Hal9000 model

Doc rewrite

Black Cat finds a classification of emotions:

Cookie sells his home to pay for more GPUs and electricity for training models:

BGM’s abridged series:

Jul 20, Cookie’s FlowTTS model

Jul 23, Blender Anon struggles with rendering XFL, which we’re doing again now

Jul 27,

>The dataset linked in this thread is probably the best emotive dataset I’ve seen and provides a very extensive range of voices, which makes it academically enticing to make use of as much of it as I possibly can for completeness.

Jul 27, BGM’s unabridged abridged series

Adorable Luna slows down for Anon

15 offers the Wheatley data

Jul 28, some guy from Discord gets flamed

Jul 29, Using punctuation to control the speech

  • https://desuarchive.org/mlp/thread/35678195/#q35678463
  • This is in- line with the direction NLP has been taking for about a year and a half now (T5 architecture). Models that take natural language as input can be controlled pretty flexibly directly from the input.
  • This isn’t new, but it’s worth reconsidering and comparing to trends in AI.

Blender Anon and I concur that getting the animation data will be an ungodly pain in the ass

Jul 30, Cookie looking into SSVAE

Jul 31, The Cocaine Saga

Aug 3, CPU speech generation

TensorflowTTS + FastSpeech seems to be fine on a CPU too.

Snoopy’s Dash and Twi in the Morning

Snoopy’s Great and Powerful Radio Show

Aug 4, Anon playing with GPT- 3

Scripted and voiced entirely by AI:

Aug 6, Comic Anon still going strong

Aug 7, Delta and I consider using audio file formats to find good audio representations

Aug 9, BGM considers making short background music clips to add to clips

Cookie training models with controllable speech and emotion

Aug 11, controlling emphasis

Aug 14, /mlp/ takes the AI General away from /g/

Let’s fly to the castle

Aug 15, voices are not protected under copyright or right of publicity

Aug 16, anon gets Festival Multisyn working:

It’d be interesting to see how well this works if we auto- generate a bunch of speech to throw into its database

Aug 17, anon on why you should clean your data

Aug 20, Blender Anon goes on hiatus

Aug 21, Woodlore’s Insults Are Magic, by Clipper

Aug 23, learning to do data stuff with Airflow

15’s first luna model

Aug 24, this guy saves me from reading 156 pages of technical documentation

Aug 28, the only reliable free tts out there

Aug 29, collecting the leaks

Aug 31, using CRC32 to deduplicate files

Sep 4, All of the leaks collected and uploaded

Cookie still trying to improve vocoder quality

Sep 6, Cookie paused preprocessing while waiting for me to finish

Cookie finished trying out AEF WaveFlow

GTA datasets from persona nerd

Sep 7, Synthbot sets up a docker container for running Cookie’s scripts

We struggle to figure out how to transfer 250+GB to Clipper

Snoopy’s last clip for a while

Delta showing off FastSpeech2:

Getting a better model: - https://desuarchive.org/mlp/thread/35919362/#q35924052

Drawings of TTS:

Sep 10, “How could you possibly be halfway done?”

Hell descends upon some anons:

Sep 11, anon dumps more datasets (Bob Ross, Nameless Hero, Witcher 3, Postal Dude, Glados, Cave Johnson, Saxton Hale):

15 adds arpabet support:

Sep 12, Shim with an old model:

Sep 14, archive of the Dazzlings + Wubcake

CPU synthesis tool from Delta:

From 15: “3 days on 8x V100s, the quality stagnated around 12 hours in”

Sep 16, Delta’s downloadable program with 48khz upsampling

Sep 18, RD plays Minecraft

Sep 23, Clipper joins in on comic transcripts

Delta adds multispeaker support for local TTS

Sep 25, Delta tries ESPNet

Sep 27, Delta tries Multi- band MelGAN

Sep 28, Refactoring my UI spaghetti code

Sep 29, Can’t believe Cookie is fucking dead, again

Another data dump, from persona nerd

Sep 30, AWS kills 15’s models

Delta hits the limits of FastSpeech2:

Oct 2, echelon being a faggot

15: “multi- speaker models are working fine”

Cookie almost done with HifiGAN implementation

15 testing torchmoji

Oct 5, Regular “is this legal”

Oct 7, Wubcake dataset

GPT- 3 can troll twitter

Oct 8, Chrysalis goes berserk (RealDash)

Oct 8, We still have no idea what to do with our animation files

Oct 11, Comparison of all of 15’s models so far:

Start of a long friendship “- https://desuarchive.org/mlp/thread/35961555/#q35984547

Oct 13, Back when AI dungeon was good

Oct 14, Cookie and 15 trying to figure out why the ngroks are fast

Oct 18, Anniversary stream ends. Holiday is over.

Idea for segmenting pony parts for image AI

Oct 18, Creating a website with React

Oct 20, None of the big three animation programs use AI

Oct 21, RealDash’s “Here For You”

Oct 22, SnoopyAnon still alive

Oct 24, Celestia responds to anons’ letter, from SnoopyAnon

Oct 25, Twilight sings a song mostly in tune (BGM)

Oct 26, Snoopy’s War of the World skit

Oct 28, First HifiGAN test

Oct 29, Cookie on how to help with models

Cookie trying 3 more vocoders

Oct 30, HifiGAN showing more promise

BGM posts music loops for anons to try out with singing voices

BGM’s Pony Zone with Trixie

Oct 31, Google breaking things in Drive without warning

BGM helping Cookie train

Nov 1, HifiGAN showing a lot of promise

Cookie going into debt for the PPP

BGM only wants to create a glorious and prosperous Equestria

Nov 2, SSVAE2 didn’t work

Nov 2, Idea to test on only the worst- performing samples

Nov 8, AWS quirks that limit 15’s quality

Nov 9, 15 offers GPUs for Cookie to use

Nov 10, Trixie’s Talk Show by BGM

Nov 11, Idea for a full Rarity episode

Nov 13, Clipper’s Mayonnaise clip

Nov 23, Website is finally ready

Nov 25, Can’t believe Cookie is fucking dead, again

Start trying to work with our animation data:

Nov 26, suggestion to use an SWF decompiler

We can just use Flashplayer + Actionscript. That seems to give us access to all the data, so there’s no need to decompile.

RTX Voice for audio cleaning

Nov 27, Decide to check out JSFL for working with the FLA files

Nov 28, Realizing the differences between 3D animation (well- researched) and 2D animation (poorly- researched)

Dec 2, Synthbot destroys friendship between Cookie and 15

Anons from /g/ come over to ask us for coding help.

Anon gets pissy about names

Dec 4, 15 starts using HifiGAN for a huge improvement in quality

Dec 5, labeling a story with audio queues

Dec 10, BFDIAnon joins with his own models using Cookie’s repo

Dec 11, Making characters whisper

Anon discusses getting scraped fanfic data:

Dec 13, Anon points out an interview where we can get Mr. Rogers data

Dec 18, Random kids on the internet taking credit for the models

New Pony Zone

Dec 19, Pony Thread Simulator

Dec 20, Fixing Big Mac data to separate Orchard Blossom

Dec 22, Getting Chrysalis’ vocal effects

Using modern JavaScript with JSFL

Switching to TypeScript: - https://desuarchive.org/mlp/thread/36313553/#q36319531

Dec 23, Explanation of how to get started with JSFL in Adobe Animate

Guided meditation with Luna

Dec 26, Slowly learning how the JSFL APIs work and how Animate structures things

Learning to dump batch renders: - https://desuarchive.org/mlp/thread/36313553/#q36332650

Dec 27, More data from GothicAnon

Dec 28, We expect we’ll need 1Tb to 10Tb for the animation data

Dec 29, Considering using Vocaloid- like voice banks

The model pronounces ponepaste properly

Jan 1, Pondering a better phonetic language than arpabet and ipa

Jan 2, Everyone’s PPP plans for the year

Jan 3, Idea to have animation AI follow the same structure as animation software

  • https://desuarchive.org/mlp/thread/36358165/#q36361775
  • “We had no idea what we were doing when we discussed this 5 months ago. We had long discussions trying to figure out how animation worked, what data might even theoretically be available, how we could reuse various animation tools to get the data, and how we could parse the data. We looked into OpenToonz, Blender, and Unity, read through a ton of manuals, tried to get JavaScript and Python libraries working, tried manually parsing the FLA files, and tried using ActionScript.” The AI tell us Animation AI is impossible
  • https://s2.desu- usergeneratedcontent.xyz/mlp/image/1609/70/1609701525741.png

Jan 4, Over 1 million audio files served within the first 24 hours of 15.ai relaunch

Jan 5, Dumping renders of symbols complete

Clipper notes (most recent 10 threads) 06 Jan 21: Synthbot Animation Update

07 Jan 21 Synthbot Animation update

07 Jan 21 Pinkie says Pog for the last time

09 Jan 21 GothicAnon makes Postal Dude dataset

Jan 10, Hacker Anon helps getting rid of Adobe Animate popups

Hacker Anon explains how to mod Adobe Animate

20 Jan: Clipper finishes sorting animation data for Synthbot

20 Jan: First attempts at generating images?

26 Jan: Anon testing BVAE- TTS on Twilight and a few non- pony voices.

31 Jan: 15.ai launches with updated UI

5 Feb: Examples of voice changer/voice style transfer

5 Feb: Synthbot patches Animate for “error opening file” popup.

8 Feb: Wubcake gets butthurt

12 Feb: "a drawing of twilight sparkle relaxing by the fireplace", Generated by The Big Sleep colab notebook, out of the box, no modifications.

13 Feb: Poopsikins makes a fully transcribed Adventure Dataset, 32 characters.

13 Feb: persona nerd makes datasets for every single character from Fallout 3, Fallout New Vegas, Fallout 4, Oblivion, and Skyrim.

16 Feb: Greentext AI anon publishes his models

17 Feb: persona nerd makes The Frontier Dataset.

17 Feb: Clipper makes first post about making an image dataset.

17 Feb: More detail on laying ground work for pony butt dataset

19 Feb: Cookie updates ngrok

20 Feb: Clipper starts labelling pony butts

21 Feb: DocAnon makes a tool for labelling pony butts

21 Feb: More updates to ngrok

23 Feb: First HiFi- GAN Ngrok

24 Feb: 15 starts character spreadsheet

02 Mar: Vocodes is pathetic

Mar 4, Render Anon writes a script to render exported Animate data

Mar 5, First bug report to Adobe

08 Mar: OpenAI CLIP has pony neurons

09 Mar: BFDIAnon uses DeepDanbooru to generate sliders for TPDNE.

15 Mar: Vocodes continues to be pathetic

16 Mar: Anon attempts to use pitch shifting for S1 Luna

16 Mar: BFDIAnon modifies ngrok with the | emotion feature

16 Mar: Update to ngrok model

09 Apr: Anon adds audio super- resolution's been added to the HiFi- GAN notebook.

09 Apr: Cookie makes a new “GANTron” model

09 Apr: BFDIAnon notes that the PPP has positively affected his life

18 Apr: Anon makes a training notebook for fine- tuning HiFi- GAN.

20 Apr: Anon tries to use StyleCLIP.

23 Apr: Delta makes a guide for using DeltaVox RS

Apr 23, Adobe fixes their bug

24 April: CliPPPer is born

25 Apr: Progress update on HIGAN decoder from GothicAnon

28 Apr: AI Dungeon goes downhill

30 Apr (I think) first post from AstraliteHeart on progress on Twi chatbot

May 1, Render Anon finds a second bug in Adobe Animate

We report that one to Adobe too: - https://desuarchive.org/mlp/thread/37024977/#q37055445

03 May: Update from Delta about RNNoise performance on VCTK dataset

06 May: iCartoonFace dataset used for cartoon face recognition research contains pony

07 May: Audacity concerns arise from purchase agreement

09 May: GothicAnon finishes Mister Roger dataset

12 May: Update from Delta

17 May: Anon finishes animation script

17 May: Black Cat comments on IPFS

18 May: GothicAnon releases his script to make training data from Audacity labels

20 May: Delta add Starlight model

21 May: AstraliteHeart updates Twilight chatbot with animation

29 May: more info from AstraliteHeart

Other... Apr 29, 2021: AI Dungeon becomes the moral police


Pony Preservation Project - /mlp/con 2021

by Synthbot

Pony Preservation Project - /mlp/con 2020

by Synthbot

Preservation Project History - 2020 to 2021

by Synthbot

Missing music

by Synthbot

Animation format

by Synthbot