ITCH across multiple systems and ease of use of CUE points and analysis

Kittmaster 1:16 AM - 14 October, 2010

I need some help understanding how to transport information across multiple machines. I'm using Windows 7 x64 on my laptop and a custom built i7 930 chassis computer. I am using the same file structure on both machines and paths. I have 32000 tracks to contend with, I have my "base" set of tracks (around 5K) all properly tagged, artted, named etc. I've been in process of moving tracks from one machine to the other. I'm having trouble understanding how the architecture how the data from ITCH is handled. I know that it all exists in the _serato_ folders in "my music".

Now my laptop is 500GB, I have an external 2TB, and the "main machine" is 1TB. I've broken the two file structure into a master folder and an import folder, master is finalized no changes ready for prime time, import is files to be fixed, named etc etc.

So here is my beef, I use the laptop to do all the file repair work while sitting on the couch, I then use the 2tb to transport (and bulk storage) to the main machine. I do all the analysis stuff on the main machine (8 cores) and all the cue point for the intro and altro stuff for the tracks I commonly use. The file and crate structure is done on the laptop. So, how do a merge the two without one corrupting the other i.e. bring the cue data and analysis data from the main machine to laptop and the crate structure to the main machine without losing the cue data and analysis data?

Where is the data located after each track is sent to "analyze files" button? What holds the 5 hot cue points for each track?

If the crate structure and file structure and paths are identical between the two machines, will machine A and B see the same files and data without losing or having to rescan all 5K files every time it is updated?

I know that may sound clear as mud, so if you you need clarification or I missed something, please just ask.

The files themselves are easy to archive, but I want to ensure I don't loose all the work that i'm doing within ITCH or have to do it twice.

Please, any advise would be grateful.

4:06 AM, 14 Oct 2010

Discussion moved to Numark and Serato

9:34 PM, 14 Oct 2010

Discussion moved to Numark NS7, NSFX, and V7 Help

sbangs 10:08 PM - 14 October, 2010

Hello, Unfortunately database merging is not a feature of the program however I understand there is open source tool to help in this type of scenario.
www.serato.com.
I will also ask if there is any other way.

Kittmaster 12:03 AM - 15 October, 2010

This should be an immediate feature addition, it is unbelievable that this isn't a core feature considering what ITCH really is. Do they really expect that only one computer is the only place that this information would reside on??

This is a huge disappointment, and goes back to my statement that vinyl jocks won't convert because of these limitations.

Thanks for the linkage

sbangs 12:05 AM - 15 October, 2010

It seems this tool is the only way for now, I will file this as a feature request.

DJdaveZ 5:19 PM - 15 October, 2010

basically, youre talking about moving songs around... once you analyze a track, the waveform and cues are saved to the track itself... not within itch... so you can move the track to your other computer or whatever you do with it, then you just have to add the new track to your library in the right spot. you won't have to re-analyze.

Kittmaster 6:11 PM - 15 October, 2010

Oh really? I wasn't aware, I will move a known track with cue points to my laptop and then open it in itch and see if the cues come across as you say. I'm assuming it is saving it as metadata like ID3 info like artist, title, bpm etc?

I'll give this a try, thanks for the feedback.

Nonnus 2:05 PM - 16 October, 2010

sorry to butt in but i am not really sure any info is actually saved within the track

i just tried different os on same machine,
tracks are loaded from same external hd
(so all info should be there, right ?)

on new system the tracks dont even have bpm info to enable auto loop,
so it does not seem this info is indeed recorded on the track file
(it also did not make any sense for serato to mess with your own files contents,
we are not simply speaking of id3 tags)

in any case it is very awkward that no one at serato has ever considered the need for a proper solution to this not so uncommon problem

likewise i did not understand why this question was even moved to numark related support (wich obviously could not provide more detailed info as the issue is regarding serato itch software and not numark hardware)

Kittmaster 2:47 PM - 16 October, 2010

@nonnus

I totally agree with you. I will try this to verify as I have not had time yet. This was originally posted in the ITCH general discussion because it is a problem that transcends all products that use ITCH not just Numark.....but then someone nuked it, I complained, they restored it, moved it, and now it's "here".

I'm just converting from vinyl and I have been leery to do so, but now I'm financially committed based upon some support I got to show me how this all works, I was up this point satisfied until I had this new issue pop up. It seems like support and general knowledge is very slow or just not present.

It seems to be sink or swim with this, and while overall it is a fairly simple system, but needs a lot more polish. Has no one ever ran into this problem.......really? Some 3rd party that is cryptic at best has to be used to do a simple thing like keeping files and folder synchronized.........come on.

Very unhappy and disappointed.

And I'm betting they'll bury this because I'm an unhappy user.

Kittmaster 4:08 PM - 16 October, 2010

I tried as recommended. So on my laptop, I nuked the _serato_ folders (no special need), reopen itch and I have a clean slate, I see in my music that it is rebuilding the directory files. Then I dragged in 3 known files from my main machine analyzed, cuepointed, and used for recording thus far. With hardware disconnected on the laptop (V7's), moved the 3 tracks on at a time to the internal player and the analysis waveform and cue points were all present. So that resolves that question.

I did a comparison of the original file (ripped 12" vinyl) to the corrected file (art, analysis, etc) and obviously the file is bigger. I opened the 3 tracks in my MP3 tagger and looked at the extended metadata fields and didn't see anything there, so ITCH must add the data somewhere within the file. Don't care where, just seems that it does.

This was done using 1.7

So it appears I have some resolution........thanks for the tip on trying the method, this should be documented somewhere.....unless I missed it.

Regards,

Nonnus 5:09 PM - 16 October, 2010

so, do you think the itch data is actually saved on the files ?
therefore to move library between machines just need to move audio folders ?

after rebooting my system my tracks seem to show bpm now and i can use auto loop
(i never get to define cue points or loop points so i depend highly on the auto loop;)

ps: not sure if you noticed but i posted a feature request for better library management on general discussion forum

Kittmaster 7:58 PM - 16 October, 2010

It has to be, because when I did the test, I had a crate of 16 songs that I was using to make a "mixtape" and then I went and copied only the files from its dir to a USB drive folder, replicated that folder on my desktop, dragged it into laptop ITCH with a blank crate and all the _serato_ folders removed, and bingo, all the analysis data waveform and cue points were all in position in a "virgin" ITCH as I described above. The laptop has never seen me set up any of those cue points, it was only done on the main machine downstairs. So since I only copied the files from the main machine to the laptop, there is no other explanation. So I would say yeah, it somehow adds it to the file.

Keep in mind, these are MP3 files, not wav or any other format. All my files are in MP3 format.

:)

Matt-C 4:06 AM - 19 October, 2010

The analyzing information is saved into the files themselves for certain.

However there are a couple of reasons it may not be:

1) The file is read only, and will need to have it's permissions changed to read+write.

2) The drive the file is on is read only. This is common when switching between operating systems when some platforms can't write to a certain drive format, but can read. Temporarily the information is kept in memory rather than written to the files, so it will appear to be written. However the best thing to do here is to make sure the drive is reformatted to suit your operating system.

The way I keep my tracks organized is to keep my music on the external drive, plug it into my Mac Pro tower, analyze the files, then when I need to gig I plug this into my laptop. This way everything I do to the tracks is always present on either computer.

Kittmaster 12:17 PM - 19 October, 2010

@Matt-C

Thanks for the feedback, so my theory is also correct and not just an accident that the CUE point data is also embedded into the file as well?

My hardcore question:

Is it safe to assume that it is not stored in the metadata tags themselves? Is it appended to the physical file? What if I want to remove that information from the file directly?

And finally, why isn't any of this documented somewhere?

Thanks for your help
Chris

Nonnus 2:05 AM - 20 January, 2011

Quote:

My hardcore question:

Is it safe to assume that it is not stored in the metadata tags themselves? Is it appended to the physical file? What if I want to remove that information from the file directly?

hi
just popping to find out if there is already a reply to this important question from kittmaster: do the audio files get any extra info added except fot id3 tags and if so how can they be cleaned up ?

Matt-C 3:58 AM - 24 January, 2011

All information we add to audio files is embedded into the files themselves.

There are other tag editing programs out there, but be aware these have been known on occasion to cause corruption or compatibility issues.

However what exact tags do you want to remove? If the information is correct this is good, if the information is not correct then simply correct it in the software.

Nonnus 4:15 AM - 24 January, 2011

matt-c

actually the info i am more intrigued about is the cue points as i assume it is not stored on standard audio tags, is this correct ?

it would also be very important for serato users to have info about wich tags are read and wrote by serato,
i have looked into docs and info but could not find related info,
only got some clues by opening same track in some tag editors and serato

Matt-C 4:28 AM - 24 January, 2011

No, anything we make (cues, loops, waveforms etc) is stored to tags of the audio file.

I am wondering why this information is needed? (This will help me understand what exactly you are trying to achieve).

Nonnus 6:05 AM - 1 February, 2011

@matt-c

it should really not be that awkward that we users try to find out exactly which data is written / appended to our audio files

obviously not asking how specific serato related data is stored
(like any secret data format used to store the cue points, loops or waveforms you mentioned)
just info about all the tags that serato actually writes to the files would be welcome :)

in my specific case,
besides regular curiosity as user i am also developing a simple tool / app to automatically tag my beatport wav files and i wanted to make sure it would play nicely with serato as it is the software i use and i had read across the forums that some tag editors could corrupt the audio files

so far so good,
serato is reading all the tags my app writes,
been testing it, editing some tags, setting up cue points and playing the tracks with no apparent corruption problems...

nonnus

Matt-C 9:46 PM - 1 February, 2011

Track overview, cuepoints, loops, BPM, gain (autogain), Beatgrids, video association, track colour. I could be missing some, however anything that you do in Scratch Live to a track in Scratch Live that is later retrieved from Scratch Live should be written.

You can of course also edit normal tag info (like name, title, etc).

As for tagging program - cool! Sounds good. I can't offer more info as you mention, but the above tags are ones that we write to.

Note that we have a special BPM tag that takes into account integers, so the BPM field for tracks we will pull it across to our special BPM field.

Also I have seen that Beatport seem to have two comment fields (? not totally sure on this, so you might want to double check). This also may cause some issue.

As for the app - nice to hear! If it's not obviously corrupting the file then it sounds like you should be all good :D

Nonnus 5:49 PM - 2 February, 2011

i already opened a file on hex editor to check the info added by serato:
from what i have noticed bpm seems to be written on normal id3v2 TBPM tag,
however i am not messing with that, i will leave it up to serato to fill the tag when analyzing the track

same thing with comments tag,
imo, the user will fill them in serato

so far no corruption problems at all, but i do want to do some intensive testing
(besides keeping backup of all converted files)

if you are interested i can send you an early version of the tool / app so you can give it a go, just let me know where to send it to :)

Matt-C 10:45 PM - 2 February, 2011

Hey man.

The BPM is written to the normal BPM tag as well as our one, so this is probably what you see.

Yeah, send the app to info@serato.com and I'll have a play :D

Cheers man!

11:00 PM, 16 Feb 2011

This help request has been closed due to inactivity.

Software help archive

ITCH across multiple systems and ease of use of CUE points and analysis

ITCH across multiple systems and ease of use of CUE points and analysis

Search Results