Tenori-On tnw sample format description

Yamaha shut down all support - let this instrument live on here!

Moderators: Kent, Paranormal Patroler, ritchiedrums

Post Reply
SampleRobot
Learning to Wiggle
Posts: 6
Joined: Fri May 29, 2020 4:58 am

Tenori-On tnw sample format description

Post by SampleRobot » Fri May 29, 2020 5:12 am

Hi there,

Are there any developers out there that already got hold of the Tenori-On tnw sample format description? Any hints would be highly appreciated. I have the voice manager installed and it might be possible to reverse-engineer the format with it. Probably Yamaha uses some kind of compression for the pcm data, so debugging could be a challenge.

Thanks and best regards,

Christian

User avatar
pikablue
Learning to Wiggle
Posts: 20
Joined: Sat Nov 09, 2019 4:13 am
Contact:

Re: Tenori-On tnw sample format description

Post by pikablue » Fri May 29, 2020 4:23 pm

You must be psychic. I'm literally looking at this right now, with the goal of writing a little utility to audition directly from TNW files.

>...Probably Yamaha uses some kind of compression for the pcm data

I have only so far experimented with 44.1kHz 16 bit data, but at least for that, the samples are stored almost exactly as 16bit stereo WAV data (signed 16 bit amplitude). Rather than interleave 2 stereo 16 bit values for L and R like in "normal" WAV files, they separate the L and R samples completely; the second channel comes later. The most tricky thing is that they encode the most-significant-nibble of the each 4-nibble sample in some complicated way dependant on some combination of the original 4 WAV nibbles that I haven't quite sorted yet.

I don't *think* that this is due to the needs of the DAC, which on the basis of its data sheet, isn't that exotic.

There's a whole load of fluff at the start (the first 0x860 bytes) which looks like data assigning each sample to a particular note, but I haven't really dug into this. There's probably sample-length stuff there as well.

I had wondered whether anybody else had come across the file header before : "ST06 SAMPLE-VCE"? What's an ST06?

Hope that helps,

..pika..

User avatar
ersatzplanet
Synthwerks Design
Posts: 6509
Joined: Fri Mar 06, 2009 1:18 pm
Location: Seattle WA

Re: Tenori-On tnw sample format description

Post by ersatzplanet » Sat May 30, 2020 12:45 pm

If anybody figures this out, the FIRST product to release is something that allows you you play the files on a computer! I have a large library of files gleaned off the net and the only way to preview them is to wait that LONG time it takes to load them into the T-O.
-James

James Husted - Synthwerks, LLC - www.synthwerks.com - info@synthwerks.com - james@synthwerks.com
Synthwerks is a proud member of the Mostly Modular Trade Association (http://www.mostlymodular.com).
Always looking to trade for Doepfer P6 cases

User avatar
pikablue
Learning to Wiggle
Posts: 20
Joined: Sat Nov 09, 2019 4:13 am
Contact:

Re: Tenori-On tnw sample format description

Post by pikablue » Mon Jun 01, 2020 12:50 pm

OK, so I have now got a small application written that can audition the sound files contained within a TNW file. It can also save them out to WAV as well if you want to mix and match.

I have written it in Tcl/Tk, as a route to trying to provide some sort of cross-platform portability, and it works fine on Windows, but I will need some help to support MacOS !

To audition a sound, the app generates a temporary WAV file, and then tries to get the system to play it. On Windows, this is OK : I use the TWAPI extension to play the sound files without spawning lots of system media players. However, this is a Windows specific library. If I didn't have this extension, one acceptable fallback on Windows is to effectively use the command line : "start temp.wav" would open the default WAV file media player and play it.

I have no idea how to do this on Mac's nor any way of testing it! Does anybody know what "command line" is needed to get a Mac to play an audio file? If so, I can add it to the script and try it. Or is that even the right question to ask? I've never ever used a Mac !

For those interested, you can get the script from here : TNW Audition Script

Best regards,

..pika..

User avatar
ersatzplanet
Synthwerks Design
Posts: 6509
Joined: Fri Mar 06, 2009 1:18 pm
Location: Seattle WA

Re: Tenori-On tnw sample format description

Post by ersatzplanet » Mon Jun 01, 2020 1:35 pm

From a search I did -
"OS X 10.5 includes a command line audio player (in /usr/bin) called afplay. This is very useful if you want to play a sound file from the command line, shell script, Automator action, etc. The /usr/bin directory is in your path by default, so you can just type afplay file.mp3 to play that file.

afplay makes use of Core Audio, so I think it can play any audio file that QuickTime supports (including mp3, aiff, wav, etc.). If any one is interested, Apple also provides the source code for this application with the Xcode developer tools. You'll find it in this folder: /Developer » Examples » CoreAudio » Services » AudioFileTools.

[robg adds: There's a very simple man page for afplay, which then tells you that help is available with afplay -h. There are a few interesting options, including the ability to play a defined (in time) segment of a file, and to play a file to a defined audio output device.]"

Haven't tried it out though.
-James

James Husted - Synthwerks, LLC - www.synthwerks.com - info@synthwerks.com - james@synthwerks.com
Synthwerks is a proud member of the Mostly Modular Trade Association (http://www.mostlymodular.com).
Always looking to trade for Doepfer P6 cases

User avatar
pikablue
Learning to Wiggle
Posts: 20
Joined: Sat Nov 09, 2019 4:13 am
Contact:

Re: Tenori-On tnw sample format description

Post by pikablue » Mon Jun 01, 2020 4:03 pm

ersatzplanet wrote:
Mon Jun 01, 2020 1:35 pm
a command line audio player (in /usr/bin) called afplay.
Thanks for the pointer!

I've tried to detect MacOSX, and included afplay in the script and updated it on the link, although I can't test it. (It hasn't broken the operation on Windows). On Windows using the "start temp.wav" approach, the script's attempt to delete the temp file is usually stymied by the fact that the player - whatever has been called up - still has the file open. I guess a similar thing might happen here. If I get any positive reports, I'll look at wrapping the script to provide a standalone executable for convenience.

Best regards,

..pika..

SampleRobot
Learning to Wiggle
Posts: 6
Joined: Fri May 29, 2020 4:58 am

Re: Tenori-On tnw sample format description

Post by SampleRobot » Tue Jun 02, 2020 5:41 am

Hi pika,

Thank you very much for your script and the information regarding the header and the samples. Very cool! Will check that out and try to find out more about the header. This already helps a lof for the implementation.

Thanks again and all the best,

Christian

SampleRobot
Learning to Wiggle
Posts: 6
Joined: Fri May 29, 2020 4:58 am

Re: Tenori-On tnw sample format description

Post by SampleRobot » Tue Jun 02, 2020 6:03 am

Just tested the script with some tnw files. Works fantastic, good job!
pikablue wrote:
Mon Jun 01, 2020 12:50 pm

For those interested, you can get the script from here : TNW Audition Script

Best regards,

..pika..

User avatar
pikablue
Learning to Wiggle
Posts: 20
Joined: Sat Nov 09, 2019 4:13 am
Contact:

Re: Tenori-On tnw sample format description

Post by pikablue » Tue Jun 02, 2020 6:27 am

SampleRobot wrote:
Tue Jun 02, 2020 6:03 am
Just tested the script with some tnw files. Works fantastic, good job!
That's good to hear. Out of interest, was that on Windows, or Mac?

In terms of the file format description, I can turn what I know into a quick text file, or Excel file, or something like that, if you are interested in digging or experimenting further. The reason I was interested is that I was trying to find out whether there is anything in the file itself that would enable users to load 16 user samples, each of which was pitch shifted by the TNR - i.e. like normal on-board TNR instruments - rather than treating the bank-of-16 as a single instrument. Most of the first 2000 bytes or so looks redundant - repetitions of the same information - which is peculiar. I also haven't unpacked this area of the TNR firmware yet to work out whether it itself is doing anything magical with any of this data.

(The encoding format for the MSB of the sample data itself is a bit of a mess of bitwise XORs and ANDs and things; in the end it was easier to just express it as a correction, as you can see in the script)

Let me know what you are after, and I'll try to oblige.

Best regards,

..pika..

SampleRobot
Learning to Wiggle
Posts: 6
Joined: Fri May 29, 2020 4:58 am

Re: Tenori-On tnw sample format description

Post by SampleRobot » Wed Jun 03, 2020 1:27 pm

Hi pika,
pikablue wrote:
Tue Jun 02, 2020 6:27 am
That's good to hear. Out of interest, was that on Windows, or Mac?
I've installed it on a Win 10 machine with the wrapper you recommended. Will check it on Mac as well and let you know.
pikablue wrote:
Tue Jun 02, 2020 6:27 am
In terms of the file format description, I can turn what I know into a quick text file, or Excel file, or something like that, if you are interested in digging or experimenting further.
That would be really helpful, thank you very much for this offer. I appreciate it a lot! Will send you a copy of SampleRobot as soon as the format is implemented.;-)

I just found out about your Tenori-On OS mods. I'm really excited to install your latest extension!

Keep up the good work and all the best,

Christian

User avatar
pikablue
Learning to Wiggle
Posts: 20
Joined: Sat Nov 09, 2019 4:13 am
Contact:

Re: Tenori-On tnw sample format description

Post by pikablue » Wed Jun 03, 2020 4:56 pm

I've added a link to an OpenDocument spreadsheet containing my file format notes at the bottom of this page: https://www.pika.blue/posts/tenori-on/audition/.

If you discover anything else (e.g. what fields, if any, indicate 8/16bit data, or 22/24/44.1/48 kHz sample rate) let me know and I'll update it. At some point I want to experiment with changing the midi note allocations, but loading them into the hardware TNR each time takes so long...

User avatar
pikablue
Learning to Wiggle
Posts: 20
Joined: Sat Nov 09, 2019 4:13 am
Contact:

Re: Tenori-On tnw sample format description

Post by pikablue » Thu Jun 04, 2020 2:49 am

pikablue wrote:
Wed Jun 03, 2020 4:56 pm
At some point I want to experiment with changing the midi note allocations...
OK, I couldn't resist. The midi note allocation field indicates what pitch the source sample is; the TNR then pitch shifts (only by adjusting the sample playback speed). If you fill a TNW file with 16 lots of the same sample, and then hand edit the TNW file to indicate that all 16 samples have pitch=midi note 60, the TNR actually still allows you to play fully chromatic/ionian scales. You have to make this change in the file itself; the Yamaha user voice manager doesn't let you indicate what pitch the original source note is.

This makes me think that the TNW header file format is linked to the AWM2 approach, and that perhaps some of the other null fields are envelope aspects that - if we could only access the binary for the existing on-board sounds - would turn out to be attack, release, etc. Do you know anything about the likely format for Yamaha AWM2 sounds?

[ Slowly creeping towards the goal of having 48 user samples, each of which is treated as a separate pitch-shifting instrument ]

SampleRobot
Learning to Wiggle
Posts: 6
Joined: Fri May 29, 2020 4:58 am

Re: Tenori-On tnw sample format description

Post by SampleRobot » Fri Jun 05, 2020 6:43 am

I downloaded the file, it's an extensive ressource! Thank you so much for all the efforts, highly appreciated! Will let you know if I discover some other parameters during the implementation. It is indeed strange that some sections seem not to be used or deactivated for the Tenori-On. When the Tenori-On came out I had the impression that Yamaha used technology from their compact QY100 music sequencer which features AWM2 sounds and a card slot (in this case smart media) for data exchange. So I bet there are ways to unlock editing of sounds also in the Tenori-On.

Best,

Christian

User avatar
pikablue
Learning to Wiggle
Posts: 20
Joined: Sat Nov 09, 2019 4:13 am
Contact:

Re: Tenori-On tnw sample format description

Post by pikablue » Tue Jun 09, 2020 8:16 am

Interesting pointer to the QY100. The chip in there is the SWX00B, which was also used in a lot of the Yamaha PSR keyboards, and which came along quite a long time before the TNR (chip in 1998, QY in 2000). However, as far as I can discover, the data on the card slot is basically MIDI files, not sound definitions; I can't find any examples of QY-100 files containing sounds.

BTW, the User Voice Manager app, if given 8 bit PCM WAV files, just pads them to 16 bit. I haven't yet tried any sample rate other than 44.1kHz, but it wouldn't surprise me if it resamples the incoming WAV files to force them to 44.1kHz in the TNW file. (It can't cope with odd rates like 16kHz).

The voice manager truncates the samples to 85504 bytes; i.e. the famous 0.97s at 44.1kHz. (And long samples get given a short "fade" at the end). You can effectively get more than 0.97s long samples by taking your original WAV, doubling the speed in an audio editor, then converting that new WAV to TNW format. Modifying the source pitch indicated in the TNW file up an octave, via a hex editor, then causes the TNR to pitch shift it back down an octave, back to the original speed. This comes at a loss in sound quality, since you are effectively halving the sample rate to 22 kHz - but it sounds OK for most things and gives ~2s samples. Doubling again, to get 4s samples at 11kHz starts to sound rather muddy...

SampleRobot
Learning to Wiggle
Posts: 6
Joined: Fri May 29, 2020 4:58 am

Re: Tenori-On tnw sample format description

Post by SampleRobot » Tue Jun 09, 2020 9:54 am

I don't own a QY100, only a QY70 so I cannot test the card data. Have to check whether there is separate sysex dump of the voice data on the QY70. I just found out that both devices sound different from each other, might be due to different chip sets.

Thanks for the info about the TNW samples' lengths, interesting trick! 22 kHz is indeed not too bad. I never tried that but might be funny to slice long samples automatically into chunks of 0.97s and sequence them one after another.

Post Reply

Return to “Tenori-On”