Audio Video Synthesis

new version 34_05_13

2011-02-19T08:49:00.001+01:00

See the updated documentation for new features and download link

New package - version 33_05_11

2010-07-07T17:25:00.001+02:00

The new package is now hosted on Google doc

http://docs.google.com/leaf?id=0B_GyI3i_xbZcNDliMzM5NDQtNjNkYy00NzViLTg5ODAtNTI2ZmUzZDc4M2M5&hl=fr

AVSynthesis version 32

2009-11-22T16:48:00.001+01:00

Score Sequencer bug fixes, time display in the rendering window (no more in the window title).

Packaged synchronized with csound 5.11

The new version can be downloaded here

AVSynthesis 31_05_10

2009-09-01T08:32:00.002+02:00

Change to the event eidtor and fix. The documentation details all the changes.

The new version can be downloaded here, code only (jar file) here

AVSynthesis version 30_05_10

2009-08-17T12:36:00.004+02:00

Many new features are been developped since the last post. The documentation details all the changes.

The new version can be downloaded here

AVSynthesis version 27_05_10

2009-03-29T12:05:00.000+02:00

Look and feel change

For documentation and download link go here

AVSynthesis 25_05_10

2009-01-17T10:11:00.004+01:00

This is an important release with a lot of changes, the main being a new GUI.

For documentation and download link go here

AVSynthesis 23_05_09

2008-10-19T10:35:00.004+02:00

Csound developer Michael Gogins has solved the DLL loading problen on Windows. AVSynthesis is working out of the box with csound double edition version 5.09.02

This new version has some new features:

Effect amount added. If amount equals 1, the input signal is totally modified. At 0, it is not transformed by the effect
AudioIn generator. This generator reads the stereo input audio device, specified by -iadc in the csound field of config.xml
Preliminary OSC support.

Documentation is here

AVSynthesis 22_05_09

2008-09-30T21:50:00.002+02:00

AVSynthesis 21_05_09 was introduced to be synchronized with csound 5.09. It has also new features

Enhanced ring modulator
New Sound Generator "CycleWave", oscilator using single cycle waveform sound files
New Sound Generator based on Vosim opcode (the vosim csound opcode is broken in the current csound package, a fix has been developped)
GUI cosmetic changes
The Real time Rendering size has the width and height of the AVSynthesis window (no more a 16/9 ratio)
Fix for Linux stability issue.

Version 22_05_09 fix a portablity bug.

Documentation

Csound Vimeo Group

2008-07-07T12:05:00.003+02:00

Ben McAllister has created a group on Vimeo for the Csound/Video users.
http://www.vimeo.com/groups/1156

Ben is working with Processing.

AVSynthesis Version 20_05_08

2008-07-07T12:04:00.001+02:00

Fix cosund thread synchronization

Documentation and instruction for download here

AVSynthesis version 19_05_08

2008-06-29T11:09:00.001+02:00

New features for the sequencer: Master clock synchronization and variable number of steps.

Documentation and instruction for download here

AVSynthesis Version 18_05_08

2008-06-28T12:17:00.003+02:00

No new features, corrections only.
MIDI Controller assignment and min-max fixes.

Documentation is here, with download link
The documentation has been updated by Dave Phillips on the use of MIDI Controllers.

AVSynthesis Version 17_05_08

2008-06-27T09:24:00.004+02:00

AVSynthesis has been updated to version 17_05_08.
MIDI Controler support lets use AVSynthesis as a real time instrument, both for the visual transformation and the sound synthesis.

Parameter value can be controlled by MIDI Control Change.
Display min-max range values.
Change navigation structure (sound synthetizer structure shown on pair along the graphical navigation buttons).
Sound and graphical rendering can be individualy activated for each layer.

See the updated documention here for download instruction. This documentation has also been updated by Dave Phillips for using AVS on Linux.

AVSynthesis version 15_05_08

2008-05-18T12:03:00.000+02:00

Add gcRecorder parameter to config.xml.
Add new Sound Generator WaveTable.
Add new parameters to Wraping and BumpArt shaders.
GUI cosmetic changes.

The documentation with download link is here.

AVSynthesis version 14_05_08

2008-05-11T13:50:00.002+02:00

Add sr and srRecorder parameters to config.xmtl.
Clean opengl code in the main display loop (source of erratic behavior).
Clean internal string variables handling.
Expose several Effect parameters that were used but not editable

AVSynthesis video introduction has been updated to this version.

The documentation with download link is here.

AVSynthesis version 13_05_08

2008-05-06T13:29:00.000+02:00

Change in csound handling to enhance stability (linux).
Add videoWidth and videoHeight
Fixes to the Recorder. Male it works correctly for a sample rate of 44100 and a FPS of 30. Wav file and number of generated frames will give the exact movie length

The documentation with download link is here

AVSynthesis version 12_05_08

2008-05-04T10:25:00.001+02:00

For solving stability issue on Linux, the csound thread design has been changed to the use of normal Java thread, in place of the CsoundPerformanceThread class provided by csound.
Fix: refresh main frame window title after bact rendering (aka wav image recorder for creating movie).
Fix: calculate the correct rendering length when a layer has a volume curve with no points.
As the jogl version is stable, and performing well, the lwjgl version is removed from the package and the documentation.

The documentation with download link is here

AVSynthesis version 11_05_08

2008-05-03T07:57:00.000+02:00

Switch back to TGA file recorder. Memory consumption seems to be solved with latest device driver. Also the code based on bmp file generation was not working on some linux environment.
JOGL is at 1.1.1 level

The documentation with download link is here

AVSynthesis version 10_05_08

2008-05-02T10:03:00.001+02:00

Change the screen shot code for movie creation, as it was causing too much memory consumption. The picture file format is bitmap.
Add the possibility to use the shift key in place of the alt key for Linux environment.
Fix bug with the RMS max value display code.

After updating the ATI grahic device driver, the JOGL version is working perfectly on Vista. This version will be the reference version that will be developed further, the LWJGL being in freeze mode.

The documentation with download link is here

AVSynthesis version 9_05_08

2008-04-25T21:44:00.000+02:00

This is a more stabilized version, with some new features

fullscreen mode implementation for the jogl version
quit button for the jogl version
fix a weird behaviour in the OpenGL code

The documentation with download link is here

AVSynthesis version 8_05_08

2008-04-19T10:16:00.001+02:00

Logic design change in the csound rendering engine for better stability
Minor UI change in the lwjgl version (sound parameter description label)
Documentation is here

AVSynthesis version 7_05_08

2008-04-17T20:03:00.000+02:00

Save button animation added to the JOGL version
Documentation is here

AVSynthesis version 6_05_08

2008-04-16T22:34:00.001+02:00

Csound thread handling correction (JOGL version)
See here

AVSynthesis version 5_05_08

2008-04-16T20:27:00.001+02:00

GUI correction (JOGL version)
Updated Documentation
Fix (pitch.dll)

See here

AVSynthesis version 4_05_08

2008-04-08T20:02:00.000+02:00

Cosmetic change only. Documentation is here

AVSynthesis version 3_05_08

2008-04-07T20:50:00.000+02:00

This a technical update. Documentation is here

AVSynthesis version 1_05_08 Video Introduction

2008-03-22T14:29:00.000+01:00

AVSynthesis version 1_05_08 Introduction from jean-pierre lemoine on Vimeo.

AVSynthesis version 1_05_08

2008-03-22T11:30:00.001+01:00

A new version has been uploaded. This version is synchronized with csound 5.08 and requires it. The major improvement is in the sound quality.
Instructions are here

AVSynthesis on Linux

2007-12-20T22:01:00.001+01:00

Dave Phillips has made AVSynthesis working on Linux. This is not a trivial task as AVSynthesis is using third party libraries, like LWJGL and Csound. Great job Dave!

He has also produced some video demos that sounds great at http://linux-sound.org/avs-examples/

Here is a thread in the Linux audio mailing list http://lists.linuxaudio.org/pipermail/linux-audio-user/2007-December/049803.html

AVSynthesis Version 1_03_07

2007-11-18T12:28:00.000+01:00

I have just uploaded a first public version of AVSynthesis for those who want to experiment.

Instructions are here.

AVSynthesis Introduction

2007-11-10T16:33:00.000+01:00

Here is a short video tour to AVSynthesis.

AVSynthesis Tour 1
envoyé par HplanKjp

AVsynthesis Composition TerraFusionScape

2007-11-10T14:58:00.000+01:00

Here is a compostion made with the last version of AVSynthesis. Sound and Visuals are rendered in real time. Csound is entirely used for sound synthesis and no samples has been used. Video quality is reduced of course for Web publishing.

TerraFusionScape
envoyé par HplanKjp

AVSynthesis

2007-05-21T21:59:00.000+02:00

Some news after a long period.

I have been busy after my PC refused to boot, and I had to quickly replace it with a laptop, pre loaded with Vista. It was in March, and XP was not an option at this time (and this is no so widely available these days). The transition was not easy as the ATI display driver for the x1700 card that equips my new laptop has no OpenGL support!

Waiting for an update, I have done some tests with Ogre3D (a C++ 3D engine) and csound (worked fine together), as well as using vvvv and OSC protocol with csound (ok too). After a while, an updated driver supporting OpenGL was released by the laptop vendor and I was pleased to use again HPKComposer. The performance are right (sufficient). Core 2 duo (1.66) and X1700 class GPU have enough power for displaying three simultaneous layers of image transformation, and for running csound in real time.

After recovering my previous environnement, I have started the development of a new software derived from HPKComposer, AVSynthesis, with the following design points in mind:

No dependency on the Eclipse frameworks, for facilitating cross-plaform distribution.
Simple and multimedia like User Interface.
Random generation of composition made of Audio Visual layers (maximum of three at any time), for ever changing runtime interpretation . The software becomes part of the composition process.

Here is a link to a document that shows some screenshots of AVSynthesis.

2007-02-19T14:51:00.000+01:00

Generator.X has an article on Thorsten Fleisch. He is a a musician as well as a filmmaker, and is using sound effects and 4D Fractal for this film Gelstat

Csound Journal article

2007-01-06T11:01:00.000+01:00

I have written an article on HPKComposer and its concept in the Winter 2007 Csound Journal.

The Csound Journal is great source of information on Csound use and users. To read them, go here

Serdar Camlica

2006-11-17T21:57:00.000+01:00

I have just discovered Serdar Camlica works via Videoart.net, so organic and synthetic!
awaken 2006 aug release animation of the year award at the 2006 digifestival.net italy.
His Web site 3dfiction.com

Das apokalyptische weib

2006-11-12T18:33:00.000+01:00

The vvvv site has some pictures on the project das apokalyptische weib. I put this information in this blog for two reasons: vvvv is a toolkit for real time video synthesis, thus it is another tool to consider, and I had with my colleague of HplanK the same performance experience several years ago. For the celebration of Manessier, Didier Debril made a composition (computer and syntheizers) and I created 3D scenes (VRML97) that were projected inside a church. Projectors were also used to light stained glass windows made by Manessier in this church.
mp3 and pictures are on line here (in French)

Reactable improvisation

2006-11-10T19:50:00.000+01:00

The reactable again. A Long improvisation that sounds like an analog modular synthetizer.

NeoFormation

2006-10-31T10:52:00.000+01:00

NeoFormation
AudioVideoPainting made with HPKComposer, no samples, no score, csound audio generators only.

HPKComposer and the Csound API

2006-10-31T10:11:00.000+01:00

Since the version 5 of csound, a rich API has been developed for use by many languages (C++, python, java, lua...). The csound developers have done a great job, and has made possible to build really sophisticated application that embeds the csound engine.

HPKComposer is based on that capability. I wanted a common paradigm for controlling the evolution of both visual and audio parameters. This paradigm is the use of looped linear segments: it is simple and different cycle durations provide an ever changing result. Csound has an opcode for doing that (loopseg) and that is fine: as sound parameters must be refreshed at a much higher rate than the graphical one, the audio engine must drive the graphical world. This is made possible by another csound 5 feature: the software bus and the capability to export a global variable as a channel of the bus. This global variable can be read by the host program using the csound API, and this is how the loopseg values are passed to HPKComposer for controlling the graphical evolution.

Thus HPKComposer real time performance program is a java program using two threads:

the master thread is controlling all the image processing and read the values generated by the csound engine
the secondary thread, created by the master thread, run the csound engine

This is working nicely, very stable and smooth.

Another interesting feature of the csound API is the possibility to advance the csound execution engine for a precise amount of samples. It is perfect for writing image buffer to disk: the csound thread run the audio engine for the number of samples required for a video frame, the main thread read the loopseg values, calculate the image and draw it to disk. This is how I produce the sequence of images that VirtualDub will transform in a video.

Reactable

2006-10-31T07:56:00.000+01:00

The reactable is a tangible musical interface: state-of-the-art multi-user electro-acoustic music instrument with a tabletop tangible user interface.

More information here

Kaoss Pad 3 controls motion graphics

2006-10-30T23:27:00.000+01:00

YouTube via matrixsynth blog. The Kaoss pad 3 provides nice visual feddback.

TerraFluid

2006-10-27T19:06:00.000+02:00

TerraFluid
This AudioVideoPainting is made with HPKComposer. Sounds are only generated with csound audio generators, no samples and no score.

HPKComposer

2006-10-27T18:26:00.000+02:00

HPKComposer is a tool I am writing for experimenting "Audio Video paintings". It is based on the Csound 5.0 synthesis engine, and on an image transformation engine, written using a java OpenGL library (LWJGL), and exploiting GLSL pixel shader .

The composition principles enforced by this tool are the following

the composition is generated in real time, both the audio and the video parts,
a composition is made of layers, each layer combining audio synthesis made using csound and image processing using pixel shader. Each layer has a transparency value that can be modulated. Linking this value to the audio volume is the primary way for creating a composition,
segment curves are used for controlling the evolution of image processing and audio synthesis parameters. These curves loop when the last point is reached and, by using segments of different lengths, will give an ever changing result,
it is possible to interact live with "Audio Video paintings" using MIDI controllers.

This tool is not publicly available, as I am short of time for supporting it, but I will post some works done with it, hoping it is a way to exchange ideas.

Introduction to Audio Video Synthesis Blog

2006-10-27T18:19:00.000+02:00

This blog is about audio and video synthesis. It is more and more feasible to use software on personal computers for generating, in real time, audio and video composition. As more composers can explore this field of mixing audio and video in a seamless environment, this blog will focus on their works and the tools they use.