Big Media Collection, Big Headache
Posted on 18th Feb 2010 at 12:23 by Harry Butler with 47 comments
As I’ve grown up, the world of consumer media has changed beyond all recognition. Whereas before physical media was the only option, MP3s, ripping and file sharing brought a whole new meaning to the words "music collection", and I built mine by ripping my CDs or grabbing MP3s from dozens of sources. As time has gone by, my knowledge has grown and I’ve changed media players, upped my quality standards, borrowed CDs from friends and amassed what I would call a healthily sized MP3 collection. The only problem is it’s in an absolute state, and tidying it up has become a serious problem.
Some of the files have full ID3 tags, some have none, some are miss-tagged. Others have embedded album art, some just have a JPG of the album art sat in the folder, and others still have no album art at all. Then there are the mis-labeled genres, the missing track numbers and the fact that every album seems to have a different file naming convention. This makes adding my music to a media library a nightmare and makes finding the tunes I actually want to listen too more a reminder of my mess than a trip down audio avenue.
So I’ve set out on a quest to properly and fully reorganise my collection from top to toe – after all, I’d do the same with my physical media collection if it was still relevant to how I consumed music. The problem is though that due to complexities of what data you want to keep, or don’t want to keep, having a application go through your collection can be dangerous. A mis-tag here, a missing track there and suddenly I’ve got five hundred mis-labeled tracks to sort. There’s also the problem that there are very few such applications out there, and most of them are only compatible with iTunes - a media player I’m not willing to use.
So doing the whole job manually seems the only way forward, but there is help from a nifty app called MP3tag. This allows you to batch edit tracks from a variety of formats and files simultaneously, and has helped enormously in cleaning up the filth from my music collection. Opening a folder containing all my Rage Against The Machine albums, I select all the files and can easily set the Artist, Genre, Album Artist and rename every MP3 in the file with a few clicks, with more options for embedding album art and correcting ID3 tags available too.
It’s not the fastest way of sorting out my music collection, but it means I get my files the way I want them, with as much information as I deem relevant embedded into the ID3 tag. After a good ten hours at it I’m now up to “S” in my collection, so hopefully it won't be too long before I can look at that “My Music” folder with pride.
How big is your MP3 collection? How do you keep it tidy? Have you gone through the tiresome task fo sorting out your media collection? Let me know in the forums.
Some of the files have full ID3 tags, some have none, some are miss-tagged. Others have embedded album art, some just have a JPG of the album art sat in the folder, and others still have no album art at all. Then there are the mis-labeled genres, the missing track numbers and the fact that every album seems to have a different file naming convention. This makes adding my music to a media library a nightmare and makes finding the tunes I actually want to listen too more a reminder of my mess than a trip down audio avenue.
So I’ve set out on a quest to properly and fully reorganise my collection from top to toe – after all, I’d do the same with my physical media collection if it was still relevant to how I consumed music. The problem is though that due to complexities of what data you want to keep, or don’t want to keep, having a application go through your collection can be dangerous. A mis-tag here, a missing track there and suddenly I’ve got five hundred mis-labeled tracks to sort. There’s also the problem that there are very few such applications out there, and most of them are only compatible with iTunes - a media player I’m not willing to use.
A simple interface but a powerful tool
So doing the whole job manually seems the only way forward, but there is help from a nifty app called MP3tag. This allows you to batch edit tracks from a variety of formats and files simultaneously, and has helped enormously in cleaning up the filth from my music collection. Opening a folder containing all my Rage Against The Machine albums, I select all the files and can easily set the Artist, Genre, Album Artist and rename every MP3 in the file with a few clicks, with more options for embedding album art and correcting ID3 tags available too.
It’s not the fastest way of sorting out my music collection, but it means I get my files the way I want them, with as much information as I deem relevant embedded into the ID3 tag. After a good ten hours at it I’m now up to “S” in my collection, so hopefully it won't be too long before I can look at that “My Music” folder with pride.
How big is your MP3 collection? How do you keep it tidy? Have you gone through the tiresome task fo sorting out your media collection? Let me know in the forums.
47 Comments
Discuss in the forums ReplyWhat I do have is as tidy as I can make it, the beauty of starting again, everything with tags and even album art built in where possible.
Usually I'm just using something like "File Renamer" (batch renaming tool), I'm still not brave enough to go in and modify the ID3 tags (mostly because I don't really use them).
And then there's always "well. yea. I cleaned them up a while ago. but now I need to do that again", because a while ago I organized most of them, but by now I got so much more music, that I just can't seem to get myself to finish organizing the new stuff.
I guess this is what comes with being a huge media junky these days.
Life hacker did an article on tagging tools a while back http://lifehacker.com/5266613/six-best-mp3-tagging-tools
ID3 tags on music I didnt extract personally seems to be messed up generally.
I'd prefer it if music players would just diplay filename and (maybe) folder name. Especially in-car mp3-radios seem to be stuck with the MP3 tag though
Xir
*and growing
for albumart i've been using albumart.org to find the art and tag & rename to copy it into the relevant folder as folder.jpg.
for ripping i've been using dbpoweramp which also applies regain values to the id tag and i used foobar2000 to apply regain tags to all my existing files.
i've even added musicip tags so i can use that if i wish
all this took ages! but now it's at the state where it's pretty quick to add new music with everything how i want it. just need to sort out genres though.
I've been trying to go through my movies now cos i just got xbmc on my xbox and a popcorn hour. queue more mass renaming. I ended up writing an hta to do bulk renames based on the exisitng filenames in a similar method to how tag & rename works - quite useful. then i came across www.therenamer.com which mostly would do everything i wanted it too. and indeed ember media manager also does the renaming for me too - ah well...
after building my new home server in nov last year i have spent tons of time organising both Visual Meda (movies / tv) and audio
i'm up to C on audio now (after completing the Movies / tv) aim to complete before the end of the year.
In my experience if you don't do it almost as soon as you download it it gets left and the problem grow exponentially.
'Media Monkey' is a tool designed to manage massive music collections, I'm using it to manage a library of nealy 250 gigs. It's fast, stable, flexible and easy to use.
It has a 'windows explorer' type of view which I use exclusively, but you can do the usual view by artist, album etc - but views like that are only good once you've tagged everything.
I use the 'Auto tag from Amazon' to repair tags on albums, inserting images etc. I then use the 'Auto organise' with my own custom rules to clean up the folder structure into /Artist/Album/1 Track name.mp3 etc.
There's also a plugin script available to batch find covers, and scripts for all sorts of things actually.
I cleaned up my entire collection using it, put everything into a "_to sort" folder, and once I'd verified an album was OK or fixed I used the auto organise to move it to the final location.
Basic version is free, but I have the lifetime 'gold' which monitors preset folders so I don't have to 'add new tracks'
Oh and it syncs with my iPhone just fine, and will convert formats / rip as you like. It's also compatible with winamp plugins (Remember winamp!)
Only 'down side' is they don't produce an OSX version and it doesn't like wine. If you're PC exclusive this won't be an issue.
I'm using a combination of MusicBrainz Picard, BonkEnc, MP3Tag and iTunes to identify, tag, embed album art and move files. Picard is an absolute lifesaver as it can lookup files based on existing meta, or do an in-depth scan of the file if meta is missing or wrong. It also moves and renames files based on customisable rules, embeds album art and doesn't mind me loading up thousands of files and letting it work over night. BonkEnc is great for converting files and it also accesses different online databases (than Picard) and moves/renames files. I used to use Format Factory for converting, but recently I'm not entirely convinced it's not malware.
MP3Tag allows me to load up 18000+ files across different folders and quickly (?) cull multiple copies of the same file/album.
I use iTunes as my media player because I love cover-flow and I havent found anything which does it better/as well. I tried Songbird but I found it clunky, buggy and sometimes plain weird. Winamp was my player of choice for years, but recently i've found it quite slow and crash-prone. Foobar requires far too much work on my part to get it to do the things iTunes does well. Zune seems interesting, but not as mature as iTunes at the moment. I like the tag editor in iTunes, especially being able to embed album art into my music from image files on my computer, but I don't like the fact that iTunes doesn't seem to embed the album art it finds itself into the file (instead it seems to save it into the itunes folder for its own use). I also don't like the fact that I have to install quick time in order to get itunes, I don't like all the extra services it adds (bonjour, ipod support etc) and I don't really like the fact that it's an apple product (and as such customisation is not really built into it.
So there you have it. :-P
completely disregarding the best free music library manager/player (itunes) is nothing short of moronic.
at least give it a decent shot before knocking it for no reason.
ohh and I will admit, about wha, 7 years ago or something I was also against itunes, wouldnt have any of that thank you very much, tried it as other producers said it was good for organising your music library and i havn't looked back since.
I only have 42Gb of music.
The website says:
TagScanner is a multifunction program for organizing and managing your music collection. It can edit tags of mostly state-of-the-art audio formats, rename files based on the tag information, generate tag information from filenames, and perform any transformations of the text from tags and filenames. Also you may get album info via online databases like freedb or Amazon. Supports ID3v1, ID3v2, Vorbis comments, APEv2, WindowsMedia and MP4(iTunes) tags. Powerful TAG editor with batch functions and special features. Playlist maker with ability to export playlists to HTML or Excel. Easy-to-use interface. Built-in player.
Brads
I use it, and it's got (built-in) pretty impressive organising tools, so that everything you add to iTunes (say you're copying across a LAN from a mate) gets copied to your libraries in an organised, regular format, which you can even customise I think.
And the built-in batch ID3 tagging isn't half bad, just a shame iTunes doesn't support FLAC files.
Also, with iTunes Agent, you can sync any iTunes playlist with any removable drive. I use it for my Android phone (never got Doubletwist to work).
So there's my argument for using iTunes ;)
It's a shame Media Monkey had a few too many glitches for me under Wine as the main UI ran quicker on a Mac than iTunes does.
Trust me on this, google media monkey and large libraries. The two are a perfect match
You probably have a crap PC then as iTunes runs fine on my current PC and has run fine on all other PCs I have owned.
Not to mention, iTunes is quite intuitive to use.
QFT. I do find that iTunes is quite slow to start, and I don't have much more than 3000 track in my library atm. That being said, I don't think Winamp or Songbird or Zune are much better in that department. You should not need a modern pc to run a media player app.
foobar does almost everything i need it to. its incredibly powerful batch file naming is probably one of its best features. mp3tag is a part of my arsenal, for when foobar cant get decent tags. album art downloader xui is the best for getting decent album art.
itunes doesnt play flac (or vorbis), is infected with ads for its store, also requires quicktime to install, cannot auto-convert on the fly to the ipod, its playlist view is filled with redundant information, cant assign keyboard shortcuts, cant switch output to non-default device, still botches the album artist tag (which wasnt even included until itunes 7), it takes up an incredible amount of resources for what the program offers, you cant customize folder structure, its a nazi when it comes to getting tracks back off your ipod....
a couple of years ago i had a list that went on and on and on, but ill stop there, i cant remember everything off the top of my head.
I know the frustrations. All of them.
I hated using Media Monkey, didn't particularly like MP3Tag either.
iTunes has the nasty habbit of not replacing duplicates, which makes it INCREDIBLY hard to figure out what's what when an entire discography is mislabelled due to my own attempts at labelling software, and pointing it at wrong folders. :/
You might want to give it another look, as it has worked very well for me - even finding tags for some weird foreign stuff I listen to. It's not perfect by any means, but it certainly helps. And, of course, you can submit information about your albums to the MusicBrainz database, thus making it better for all.
done.
iTunes is actually functionally very good; it's just a shame it's a bit of a resource hog and insists on doing things 'the Apple way' a lot of the time. It's also a crying shame that it doesn't support ripping to (or even reading) FLAC - I would almost definitely switch to it for ripping then - the artwork it grabs is almost always higher quality than what you can usually find on google (extremely lucky to get 600x600 artwork, which is often just upscaled from poor quality scans anyway). Yes, I know about Apple Lossless. No, I couldn't use it even if I wanted to.
To complicate thing in my case the bulk of the collection I reipped to ,ogg, though I have some wma, mp3 and aac and lately I have been initially ripping to flac. I then want to have the colection replicated as needed for different devices. At hope I use a Squeezebox, which can play any format, but my personal player is a Cowon which while it can do all formats, I would rather covert to ogg as its only 30gb. Then in the car I can use an iPod or a USB disk so I need to convert everything to MP3 except the WMA!
there's plenty of server space (WHS) so I will create a master store split by codec and then write a bit of code to create separate collection best for the player, the squeezebox and the car. (I'm a developer so a bit of code to shuffle the files is fine, but I'd need to be a network admin to creat an AD structure of virtual directories so I didn't need to duplicate!)
EDIT, actually to save a little space I'll create a master collection of "best" versions (flac>ogg>mp3) which will do for the squeezebox and store the alternates elsewhere.
i've used it for years, never had a problem.
until i got my squeezebox that is. then you realise how a media player should be made! supports so many more tags, id3v4(incidentally to the chap warning about foobar it seems to use v4 which then causes some apps to have problems with the files although not squeezecentre) replay gain per track and album, musicIP plugin, proper albumartist support(incredibly annoying on the ipod), links into lastfm, internet radio etc. i find for playing music on my pc i mostly use softsqueeze so i can hook straight into the squeezecenter although i have also got itunes connected to my media share on my server. the only fly in the ointment is that i have an ipod so most of the hardwork done for organising things doesn't translate over to that
iPeng on the ipod touch is a great way to control the squeezebox though!
Most of the tagging on my files has been done by hand originally, but I use Audiograbber to rip CDs (it's old, but it still kicks arse), and MP3BookHelper to do all my tagging (odd name, but works well, and seems to be more configurable than most of the other programs I've seen).
Anyway, my collection is currently organised into <album>\<>trackno-<artist>-<track>.ext and I've got something like 40-50 Gb (not checked for ages). Most of that is ripped from CDs, some is "acquired", mainly from friends rather than p2p.
Generally I have the files on my desktop, but I also have a backup on my FreeNAS server, which is shared with everyone (when it's not forgetting it's IP address, but that's another rant).
I am planning on re-organising it all into <artist>\<album>\file.ext, but I'm holding out for a tool that can do it automagically. If anyone knows of a program that can take a whole bunch of properly tagged mp3's, flacs and oggs and spit them out into a new directory structure then I'd love to know about it.
I write a piece of software called 'bliss' which is aimed precisely at organising large digital music collections.
When people talk about managing large collections of music they always/often talk about how complete the tagging is. E.g. the fact that they don't have tagged music, they don't have album art, they're missing compilation tags, and so on. That's important, but I also think consistency within the collection is important. For example: (and you gave this example Harry) having all your art stored in a consistent way, having all the genres in your library being constrained to your desired level of granularity (for instance: there's no point having one album classified as 'rock' and another as 'post acid thrash metal revival' - how does that help?).
So this is what bliss is - a way of not only completing your tags but also getting consistency between them inside your collection. It does it by inverting how you work with your music - instead of editing music files you define your rules declaratively (how you want your music organised) and bliss does it for you, asking you for confirmation where it isn't sure. If you want it to it can run non-stop (for instance on a home server) and applies the rules when you add new music. And here's where I disagree a little with one thing you said:
"The problem is though that due to complexities of what data you want to keep, or dont want to keep, having a application go through your collection can be dangerous. A mis-tag here, a missing track there and suddenly Ive got five hundred mis-labeled tracks to sort."
This would indeed be a problem, but I think a lot of it comes down to being sensible. There are a lot of tools out there that are way too optimistic, I agree. But with 1000s of albums in a collection I think you have to have some level of automation. The key is that the software must 'be sensible' about what data to update automatically, and also provide easy ways to back out changes that have been made (one thing I think would be useful is history of your music collection).
The website is http://www.blisshq.com
@da_gravell: Looks like a nice product you have there. However, with so many top-notch open-source software available I very rarely part with my hard-earned. Other than Windows and games, I run only free/open-source software. On the bright side, I hardly think I'm representative of the average computer user :-P
That weird foreign stuff is probably quite popular in its country of origin.
Trying to retag later is a nightmare, no matter the tool.
I still have a few mis-tagged and untagged songs but I correct them as and when I listen to them.
So an i7 920 is regarded as crap these days is it? Bummer.
iTunes is a dog when compared to the alternatives - and there are very good alternatives created because a lot of people come to the same conclusion as me, and that is that iTunes is not that great a music manager.. in the same way that quicktime is not that great a media player.
Yeah, iTunes runs "fine" on my decent specced PC and my decent specced Mac, but the alternatives run much much better!
Yes I think this is what I used, or it used to be named just musicbrainz I think. It Revolutionized my mp3-collection, but as Harry said, I did get lots of mistagged mp3's. The first time I used this program I didn't even know this was a problem, so I didn't fix manually. Lots of files with wrong names :/
Biggest changes I did was putting each band and album in separate folders. Making browsing much faster, not needing to load all files at once from the harddrive. Have around 50GB now, but mostly uses spotify for musicneeds.
What the future brings, I don't know. Guess I will stick with my mp3-collection for a long time, having backups on atleast 3 computers and two external harddrives. If I get rich someday I will try to purchase all my playlists on Spotify, But I don't believe in bying premium anyway. It's like paying for your car to drive around even when you're not in it. Same with having spotify Premium and not listening all the time, waste of money :P
Best purchase I ever made as I usually clean the recording up or adjust the sound to my liking while I'm at it.
I use CDex for ripping, but I guess everyone uses something else :D
Pirated music is not a lost sale (think of it as "postponed Purchase")
Then I bought an iPod. I tried alternative ways of transferring songs to it, but found it all to be too much messing about, so I reluctantly started using iTunes. iTunes and I are simply inseparable. Even if I found a brilliant alternative that would allow me to easily sync my iPod, I still couldn't use it: it would never pass the GAF! (Girlfriend acceptance factor :)). Since we moved in together, we now have one unified music library and we both have iPods. Changing now would be more effort that it's worth.
It *is* a resource hog, but frankly is that really a problem these days? It might take, say, 10 or 20mb more memory than other apps, but I've got 4gb to spare (not to mention the pagefile). Yes, it does also start up a little bit slowly, but it takes no more than 5-10 seconds on my PC (which is not exactly high-end) - it takes longer than that to load and log in to Steam. People might say that there are "better" media managers around, but I say: "different horses for different courses". I, personally, cannot now live without it.
Sort my iTunes collection by album name. If I have duplicate tracks across different albums, that's fine - they're complete albums. A quick check tells me that I have 13 different copies of "Walk" by Pantera, and 18 copies of "Symphony of Destruction" by Megadeth. However they're all on different albums, or live albums, or B-Sides, etc... In fact, according to iTunes, I have 2618 "duplicate" items - but they're not duplicates to me.
No comments on my music taste please! Though you'd most certainly be wrong to take the piss....! ;)
MP3Tag for me, quick and easy.