Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us $3,400 per month for bandwidth bills alone, and since we don't believe in shoving popup ads to our registered users, we try to make the money back through forum registrations.
  • Locked thread
steeltoedsneakers
Jul 26, 2016


I project videos over a friend who plays music.

Recently it's gone from projecting pre-made clips over one friend, to projecting live mixed clips, loops and effects over two.

I'm creating the thread to see what similar video projects people have and to get some tips on where I should focus my efforts on skilling up - and in particular creating responsive videos that sync up to the performers' audio.

This is the kind of thing I make when given enough rope: https://www.youtube.com/watch?v=eYIy9SSdx2o

I want to know what sort of software is going to help me work with a similar aesthetic live, and not fall into the trap of it looking like a 90s dance party. Currently I use TouchViz on iPad - but I'm exploring TouchDesigner as it looks gentler than vvvv has been to me.

Have:
iPad, a couple of low spec win/linux laptops, and a mid-spec windows gaming PC (gtx 660, i5), a 3300 lumen projector.

First priority for me is the look and skills to explore new aesthetics, mapping isn't something I want to delve into quite yet.

Adbot
ADBOT LOVES YOU

steeltoedsneakers
Jul 26, 2016


Update - Iíve thrown my lot in with TouchDesigner.

Weíve got a gig coming up next month, and Iím sticking to TouchViz on iPad for that. Iíve hooked up a Korg NanoKontroller2 to it (via laptop/rtpmidi/wifi) to give myself a bit more control and quick access to a fade-to-black.

In the meantime Iím putting together a custom interface in TouchDesigner that lets me use some of the key effects/distortions I was using in TouchViz:
  • RGB shift
  • Light tunnels
  • Pixelation
  • Shuffle
  • TV lines/interference
  • Cross Processing
  • Glow
  • Stereo image
  • ASCII filtering
  • Vignetting

A lot of those are colour tweaks, so easy enough to set up. Some will be mucking around with GLSL conversions. The overall goal/hurdle is combining them with an audio-reactive element that I can alter (probably increase or decrease intensity) on the fly. Iím also bringing in some of the tricks I was using in Vegas Pro - like pixel wrapping the video around a glitch pattern to create distortion.

Iíve built an RGB shift that responds to audio, by running filter passes on incoming audio to make it only respond to bass. The effect being that the Red/Blue (or Yellow/Blue, or Green/Purple) channels will be distorted or shifted based on the intensity of the audio, while the base image/movie remains undisturbed. Pic is effect running over B&W photo.


The issue I have is that once you get outside of deep bass, even using patterns or noise to flatten out the curve, TouchDesigner uses the shape of the actual wave to manipulate images - so Iím still trying to translate that into simple horizontal or vertical motion rather than having a big olí waveform running through it.

Iím also making a concerted effort to curate the video samples Iím using. Iíve got a few VJ loop-feeling things, but Iím trying to thin them out. Mostly, Iím using weird clips (e.g. an octopus-shaped kite in slow motion), time lapse footage, scans of old, out of focus film photos and more solid, geometric feeling loops (like the thick-lined/pastel stuff Beeple does).

Will aim to put up some clips from the gig next month.

cubicle gangster
Jun 26, 2005

magda, make the tea


Thanks for making this post - I enjoy playing with video and want to have a go at this.

I've made a bunch of videos in the past that have visual elements generated by music, but they are not real time.
https://vimeo.com/132115954
I made this for a friend - he sent me stems of the bass, drums, guitars, vocals etc and I used the volume of each to control the discharge rate of a smoke simulation, so there is a separate layer & shape for each instrument. It took 3 weeks to simulate and render and by the end I wasn't particularly happy with how hard it is to really see the elements are being triggered individually (if I didn't tell you you'd probably never notice), but he was happy and I was too busy to spend more time on it. I have done a couple others as tests for my own music, but then never finished the song to do it for real. I have a bunch of better, cleaner setups ready for some music though.
I guess this is the complete other end of what you do - I make a very simple setup that takes a few hours then walk away and let it slowly generate things, with only a vague assumption of what it's going to look like when it's done.

sebmojo
Oct 23, 2010



Legit Cyberpunk

cubicle gangster posted:

Thanks for making this post - I enjoy playing with video and want to have a go at this.

I've made a bunch of videos in the past that have visual elements generated by music, but they are not real time.
https://vimeo.com/132115954
I made this for a friend - he sent me stems of the bass, drums, guitars, vocals etc and I used the volume of each to control the discharge rate of a smoke simulation, so there is a separate layer & shape for each instrument. It took 3 weeks to simulate and render and by the end I wasn't particularly happy with how hard it is to really see the elements are being triggered individually (if I didn't tell you you'd probably never notice), but he was happy and I was too busy to spend more time on it. I have done a couple others as tests for my own music, but then never finished the song to do it for real. I have a bunch of better, cleaner setups ready for some music though.
I guess this is the complete other end of what you do - I make a very simple setup that takes a few hours then walk away and let it slowly generate things, with only a vague assumption of what it's going to look like when it's done.

That's really nice. I agree the moment to moment connection with the soundwaves isn't obvious, but there's a nice flow and change as as the sounds change over the song. Is it all smoke samples, or are there images mixed in too?

sebmojo fucked around with this message at Sep 20, 2017 around 03:51

cubicle gangster
Jun 26, 2005

magda, make the tea


There are images too - filmed a bunch of clips of water at varying heights and zoom levels to fade between as sections changed. My only regret is the one with the waves crashing is too obvious, I should have kept them all calm.

steeltoedsneakers
Jul 26, 2016


cubicle gangster posted:

I guess this is the complete other end of what you do - I make a very simple setup that takes a few hours then walk away and let it slowly generate things, with only a vague assumption of what it's going to look like when it's done.

I reckon the concepts are pretty much the same though - both convey some sort of visual reaction to audio, but I've got a lot fewer GPU cycles to spare to stay reactive in the live environment so things probably need to be a bit more lo-fi by necessity.

Agree with mojo that the moment to moment might not be super visible, but the overall flow works well with the track. I'm still working out how obvious to make the connection, and some effects the less one-to-one relationship probably works better.

If you don't have images mixed in, then this is a really nice effect, there's a couple of times when you get the feeling of an image about to reveal itself - but it never quite does.

(e: saw your post about water images - there's a bit where i thought there were going to be people images.. randomness is great for these things, audience does half the interpretive work for you.)

Adbot
ADBOT LOVES YOU

cubicle gangster
Jun 26, 2005

magda, make the tea


steeltoedsneakers posted:

there's a couple of times when you get the feeling of an image about to reveal itself - but it never quite does.

yeah i'm really happy with how this part worked out actually - the song is one my boss wrote about getting over his addiction to ambien so I wanted to try and invoke the feeling of trying to remember something that may or may not have been a dream.

I wish someone would pay me to do this, it's way more fun than real work and I want to spend more time on it.

  • Locked thread