Saturday, April 14, 2018

CD - RIP

Anybody out there still ripping CDs?  Or is it just me. ;)



Recently I figured that a track was missing. No idea when and why this happened. 

OK. Quick decision taken. ReRip that CD. 
Yep - I still keep my CDs in the attic. 

Gee Wiz... ReRipping a track "quickly" turned out into a major exercise. 

That's the story about this exercise.





(Note: You'll find a HowTo at the bottom of the article!)

Intro  

I've been using dBPoweramp  (dBP) for the job in the past. It used to be my reference. And I think it still is the reference tool out there.

I havn't been ripping CDs for a couple of years though. 

I do have dBP installed on a virtual Windows10 client on my Ubuntu machine.

I quickly figured, I'm running an outdated dBP version.  As usual everytime I want to  use it once in a while I'd have to pay $30 for an upgrade!?!? 
Hmmh. It's just one track I need. Nope - I am not willing to pay anything for it anymore.  
The issue solved itself quickly.  I figured trying to make use of dBP would be useless anyhow. Why's that?

The cdrom-drive is not properly recognized by it's HW identifier through Virtualbox and therefore dBP fails to  set the right "AccurateRip" drive-offset. "Accurate Rip" a must have feature to me. Using Accurate Rip is the only way to guarantee that your rip isn't flawed. There's IMO no way to let that safety-net alone!
The underlying dBP issue: 
There's no way to enter the Accurate Rip required drive-offset manually into dBP. 
That's really bad. After getting a rather bitchy feedback of the dBP designer I put that tool aside.  IMO it's a no-brainer for the designer to add such a feature.

No, after this  I didn't feel motivated to give EAC under W10-Virtualbox a try either. 

What a mess!

Those of you who read this and are running a Windows machine ( not on a virtual machine though!) should still have a closer look at dBP or EAC. These are IMO still the first choice CD-rip apps under Windows.

Ok. Let's have a look at Linux. ( I do think Windows users might also find some hints below!) 

I kept going.

Next question. How is the situation on the Linux ripper market? It has never been great.
And I doubted that the market evolved a lot. I rather expected that the situation got worse.

I wasn't wrong with that prediction...

There are/have been several, some of them quite promising, projects out there, such as

ripperX, Rubbyripper, Morituri, Asunder and some more...

I learned, these are all pretty much dead - since years - by now. 
Some projects are still alive.  The most advanced of the bunch is probably ABCDE
However. 
ABCDE is not offering Accurate Rip support. It allows to apply a drive-offset though!
You'd need another tool that verifies the rip results against the Accurate Rip database.
Nope. That won't work for me. That'll be getting much too complex.

Again. What a mess!!!

I then stepped over my potentially final chance - a fork of Morituri called Whipper.
It seemed to be maintained. Latest GIT commits dated a couple of days back! That looks promising.

OK. Decision taken. Let's try Whipper.

Whipper comes with two - to  me -  key features:

1. Accurate Rip support
2. Automatic tagging 

It's a command line tool. Fair enough!

Whipper offers MusicBrainz tags only. Hmmh. Bad luck.
However. Whipper at least tells you if there is a CDDB entry and the CDDB id of your CD. 
It won't let you choose between MusicBrainz or CDDB tags though. 
MusicBrainz tags are supposedly of better quality then CDDB tags. 
On the other hand the MusicBrainz database is much smaller then CDDB. 

I read a Whipper user question why Whipper is neglecting CDDB tags
His opinion: "...better a low quality tag, than no tag - please, give the user the choice".  
I couldn't agree more to that! Though I doubt that we'll ever see that feature being implemented.
Just to mention it.  I don't even think that MusicBrainz offers a much better, if any better, tag-quality then CDDB at all. 

I kept going.

On github I then saw numerous Whipper issue reports
Many of them still in progress or enhancements. 
Something like that - huge backlogs - is usually not a good sign. 

Anyway. I thought - keep going!

I then tried to install Whipper manually from sources. I failed. I couldn't make it work. There are a lot of dependencies involved in the project. Usually that's not a good sign. With such a high complexity the chance of errors increase dramatically and you might end up putting days into such a project.
OK. Another hour wasted. I didn't really intend to spend more time at this point on this manual-build-from-sources task.  


Anyway. I thought - keep going!

Luckily I found a repository for Ubuntu. 
Last entry from January 2018 (Update: July-2018 - Bionic support has been introduced). 

Finally!!! 

That looks promising. Let's give that a try.

Guess what!?!? It worked! A 5 minute exercise.

So far so good. At this point I could have called it a day.

I kept going.

Now the fun parts starts. 

***********************************************************************************************

The ripping!


Where to start?

Next I was hit by the pretty much non existing Whipper documentation. 

What a mess!


Anyway. I thought - keep going.

Using Whipper Help  - helped a bit. Googling Whipper didn't help much. 
Googling Morituri did help a bit. 
The trial and error approach also helped a bit. 
And looking inside the code also helped a little.

I figured:

* Whipper rips flacs by default. Good. Check.
* Coverarts I'll fetch myself. That's not a big deal. Check.
* You  lookup the offset yourself. Fair enough. Check. 


Hint: Don't try to use the pretty useless "offset autodetect feature" of Whipper - it'll drive you nuts!


I kept going...

...and then after more than half a day of annoying digging and trying - I managed to get a Accurate Rip verified CD ripped and tagged - with flac compression level 0 - under Linux.

The (MuiscBrainz-) tagging was once more all but perfect. You for sure need to use a tag-editor to get your tags properly shaped up.

I strongly recommend puddletag for the job.  It's one of the rare Linux tools which is very well made.

The nice thing about puddletag: You can even fetch and apply CDDB tags based on the CD
raw data. Meaning. If MusicBrainz doesn't offer tags, you can load the untagged flacs into puddletag and apply CDDB tags afterwards. 

WrapUp

Looking at the bumpy road I've been walking during the day, I hope next time it'll 
take me just 10 minutes to get a CD flawless respectively "accurate" ripped under Linux.


To spare you (and me - next time) some effort and time I've written up a little Whipper HowTo to be used on Ubuntu systems. 

It'll be command line work from a terminal only. Mainly copy/paste will do!

*******************************************************************************************************
Whipper HowTo (Ubuntu)

Process:

1. Install whipper
2. Find out the drive-offset of your drive
3. Define your target formatting - directory and files
4. Rip your CD
5. Rework the tags with puddletag
6. Fetch the coverart by running Google images search 

***************************************************************


1. Install whipper


############

sudo add-apt-repository ppa:spvkgn/whipper
sudo apt-get update


sudo apt-get install whipper

#############

Note: For those who want to generate flacs with compression level 0, have a look
at the Annex 1 now!


Note: There've been issues with the conf file parser lately. It's better to use Whipper with commandline options only. 

To make sure there's no conflict you might delete a potential config file by running:

#############

rm ~/.config/whipper/whipper.conf

#############


2. Drive-Offset

Look up your drive model and search for it on Accurate Rip.
Note: A positive offset will be used without sign later on!

3. Formatting

These are your formatting options for whipper:

**** track file variables  - applicable for the audio track file only

%t: track number
%a: track artist
%n: track title
%s: track sort name

Note: The extension .flac will be added automatically.

***disc variables applicable to cuesheet .cue, playlist .pls, logfile .log  + directory structure

%A: album artist
%S: album sort name
%d: disc title
%y: release year
%r: release type, lowercase
%R: Release type, normal case
%x: audio extension, lowercase
%X: audio extension, uppercase

My choice of variables for the directory and track is:

%A-%d-%y/%t-%a-%d-%n

Example:

Norah Jones-Come Away With Me-2002/01-Norah Jones-Come Away With Me-Don't know why.flac

The other files (pls,cue,log) need the same directory setup and whatever other variables:

%A-%d-%y/%A-%d-%y

Example:
Norah Jones-Come Away With Me-2002/Norah Jones-Come Away With Me-2002.pls


Now, you're ready to rip.


4. Rip your CD

Fill in your data in below variables and then copy/paste them into the terminal:

Example: 
ACR DriveOffset for Lite-ON eBAU108-1 = 6
ACR KeyCDs are listed here

I use "Norah Jones - Come away with me" as key and test CD. It's listed on ACR as keydisc.

########################################################################

OUTDIR="/tmp/newrip"
WORKDIR="/tmp"
DRIVEOFFSET="6"
TRACKFORMAT="%A-%d-%y/%t-%a-%d-%n" ## naming for target dir/audiofile
DISCFORMAT="%A-%d-%y/%A-%d-%y"          ## naming for target dir/.log/.pls/.cue

########################################################################

And now the actual rip command:
Below lists a single command line! Just copy/paste it.

########################################################################

whipper cd rip --offset="$DRIVEOFFSET" --unknown --output-directory="$OUTDIR" --working-directory="$WORKDIR" --track-template="$TRACKFORMAT" --disc-template="$DISCFORMAT"

########################################################################

Note: The output-directory will be created by whipper!


Now you can move the new audio directory, to be found under OUTDIR = /tmp/whipper in above example, to your audiofile tree.


Task 5. and 6. of the process go beyond the scope of this article.



Some basic hints supporting your CD extraction project:

* I'd suggest to use flacs as target format with compression level 0 ( see my flac articles)
* you should look for highest quality images as coverarts via Google image search
  Look for clean images with a minimum resolution of 500x500 pixels. 
* Make your choice for the coverart filename - and then keep it that way. I use "folder.jpg" for all of them. 
* Usually you can't or don't want to use the default tag and file structures offered by
   whatever tools you'll be using. Please have a closer look at that!!!

   Use mp3tag under Windows or puddletag under Linux to add your tags.
* Have a closer look at the "genre" tag. To me this is a very important tag in dealing with my quite large collection.
   In 99% of all cases where I'm not looking for a specific album I first select the genre and 
   then the album underneath.
   Very often it the Genre tag is not set properly! 
 * Folder/file structure 
   Below you'll find my structure: 

   /music/folk/Norah Jones-Come Away With Me-2002/02-Norah Jones-Come Away With Me - Seven Years.flac

   That's the result of trying this or that over years.

* Classical Music Tagging
   That's a challenge with above structure, which is actually based on what's being offered by
   the databases and tools. You better stick to it! You actually can't get around it.
   Otherwise you might end up with compatibility issues depending on what player app you'll
    be using.

   CDBB or Music Brainz won't get you proper or consistant tags for classical albums.
   There is no way around to 
manually work on the tags!
   Just a hint. The way I do it. I add the (to me) key artist - soloist or conductor or orchestra - depending of the album into the artist tag. Usually the conductor/orchestra/composer goes then into the album tag.
   Make sure you have a great coverart that pretty much explains it all! You'll appreciate it
   later on.

*  Do a test run with a couple of different album rips and tools to make sure you can handle 
   the whole thing properly. You don't want to re-rip or re-tag hundreds of albums! 

########

I hope you'll manage to make it all work.

Good luck.


Enjoy.



######################################################################

Annex 1  

whipper and flac compression level

whipper applies the default compression level 5 of the flac conversion tool to the tracks
It's hard-coded in the whipper code!
CL5 is not my preferred choice. I usually go for CL0. 
I'll show you quickly how to make  CL0 the default compression level of whipper.

It's just a one-liner you have to execute after installing whipper :

########################

sudo su 
grep -i "compression-level-0" /usr/lib/python2.7/dist-packages/whipper/program/flac.py || 
sed -i "s|'--silent',|'--silent', '--compression-level-0',|g" /usr/lib/python2.7/dist-packages/whipper/program/flac.py

###########################

Note: After new whipper installations or updates you have to redo this task.


And now you can enjoy whipped flacs at compression level 0. ;)



Annex 2

Whipper  (latest 0.70 under Ubuntu Bionic) output:


I attached a 100% successful log of my test disc. Just as a reference. 

########################################################

klaus@nuc:~$ whipper cd rip --offset="$DRIVEOFFSET" --unknown --output-directory="$OUTDIR" --working-directory="$WORKDIR" --track-template="$TRACKFORMAT" --disc-template="$DISCFORMAT"
Checking device /dev/sr0
eject: CD-ROM tray close command failed: Input/output error
Reading TOC...
CDDB disc id: b90a950e
MusicBrainz disc id 1QYnhv0RV6aSgltrgNxa22bDGgA-
MusicBrainz lookup URL https://musicbrainz.org/cdtoc/attach?toc=1+14+203342+150+14112+25007+41402+54705+69572+87335+98945+112902+131902+144055+157985+176900+189260&tracks=14&id=1QYnhv0RV6aSgltrgNxa22bDGgA-
Disc duration: 00:45:09.226, 14 audio tracks

Matching releases:

Artist  : Norah Jones
Title   : Come Away With Me
Duration: 00:45:30.036
URL     : https://musicbrainz.org/release/24c72a46-4c0d-3024-ae45-3bf846893356
Release : 24c72a46-4c0d-3024-ae45-3bf846893356
Type    : Album
Barcode : 724358206722
Cat no  : 7243 5 82067 2 2

Artist  : Norah Jones
Title   : Come Away With Me
Duration: 00:45:30.036
URL     : https://musicbrainz.org/release/506fe9cf-29c6-4318-9070-da9463f51617
Release : 506fe9cf-29c6-4318-9070-da9463f51617
Type    : Album
Barcode : 724353860929
Cat no  : 7243 5 38609 2 9

Artist  : Norah Jones
Title   : Come Away With Me (Deluxe Edition) (Disc 1 of 2): Come Away With Me
Duration: 00:45:30.036
URL     : https://musicbrainz.org/release/a0a8beba-5b3c-37fe-bb01-50527bf72991
Release : a0a8beba-5b3c-37fe-bb01-50527bf72991
Type    : Album
Barcode : 724359568904
Cat no  : 7243 5 95689 0 4

Artist  : Norah Jones
Title   : Come Away With Me
Duration: 00:45:30.036
URL     : https://musicbrainz.org/release/c88138cf-00f3-3c70-8757-3bbe0e4713d1
Release : c88138cf-00f3-3c70-8757-3bbe0e4713d1
Type    : Album
Barcode : 724353208820
Cat no  : 7243 5 32088 2 0

Artist  : Norah Jones
Title   : Come Away With Me
Duration: 00:45:30.036
URL     : https://musicbrainz.org/release/e3dc76d3-2ffc-39af-b847-bbc2ccb551f7
Release : e3dc76d3-2ffc-39af-b847-bbc2ccb551f7
Type    : Album
Barcode : 724359569109
Cat no  : 7243 5 95691 0 9

Artist  : Norah Jones
Title   : Come Away With Me
Duration: 00:45:30.036
URL     : https://musicbrainz.org/release/f3d38327-9a15-38a4-9dc9-30ffc7b2d33d
Release : f3d38327-9a15-38a4-9dc9-30ffc7b2d33d
Type    : Album
Barcode : 724353208820
Cat no  : 7243 5 32088 2 0

creating output directory /tmp/newrip/Norah Jones-Come Away With Me-2002
Ripping track 1 of 14: 01-Norah Jones-Come Away With Me-Don't Know Why.flac
CRCs match for track 1                    
Peak level: 32357
Rip quality: 100.00%
Ripping track 2 of 14: 02-Norah Jones-Come Away With Me-Seven Years.flac
CRCs match for track 2                    
Peak level: 30456
Rip quality: 100.00%
Ripping track 3 of 14: 03-Norah Jones-Come Away With Me-Cold Cold Heart.flac
CRCs match for track 3                    
Peak level: 30071
Rip quality: 100.00%
Ripping track 4 of 14: 04-Norah Jones-Come Away With Me-Feelin' the Same Way.flac
CRCs match for track 4                    
Peak level: 30548
Rip quality: 100.00%
Ripping track 5 of 14: 05-Norah Jones-Come Away With Me-Come Away With Me.flac
CRCs match for track 5                    
Peak level: 30462
Rip quality: 100.00%
Ripping track 6 of 14: 06-Norah Jones-Come Away With Me-Shoot the Moon.flac
CRCs match for track 6                    
Peak level: 30546
Rip quality: 100.00%
Ripping track 7 of 14: 07-Norah Jones-Come Away With Me-Turn Me On.flac
CRCs match for track 7                    
Peak level: 30549
Rip quality: 100.00%
Ripping track 8 of 14: 08-Norah Jones-Come Away With Me-Lonestar.flac
CRCs match for track 8                    
Peak level: 30095
Rip quality: 100.00%
Ripping track 9 of 14: 09-Norah Jones-Come Away With Me-I've Got to See You Again.flac
CRCs match for track 9                    
Peak level: 30532
Rip quality: 100.00%
Ripping track 10 of 14: 10-Norah Jones-Come Away With Me-Painter Song.flac
CRCs match for track 10                    
Peak level: 30546
Rip quality: 100.00%
Ripping track 11 of 14: 11-Norah Jones-Come Away With Me-One Flight Down.flac
CRCs match for track 11                    
Peak level: 32357
Rip quality: 100.00%
Ripping track 12 of 14: 12-Norah Jones-Come Away With Me-Nightingale.flac
CRCs match for track 12                    
Peak level: 32357
Rip quality: 100.00%
Ripping track 13 of 14: 13-Norah Jones-Come Away With Me-The Long Day Is Over.flac
CRCs match for track 13                    
Peak level: 30074
Rip quality: 100.00%
Ripping track 14 of 14: 14-Norah Jones-Come Away With Me-The Nearness of You.flac
CRCs match for track 14                    
Peak level: 29332
Rip quality: 100.00%
track  1: rip accurate     (max confidence    200) v1 [6ba01a43], v2 [979d3121], DB [6ba01a43, 979d3121]
track  2: rip accurate     (max confidence    200) v1 [2a28e248], v2 [aeffedaa], DB [2a28e248, aeffedaa]
track  3: rip accurate     (max confidence    200) v1 [7529437d], v2 [7d937003], DB [7529437d, 7d937003]
track  4: rip accurate     (max confidence    200) v1 [822a4da0], v2 [adb25820], DB [822a4da0, adb25820]
track  5: rip accurate     (max confidence    200) v1 [ebc715e6], v2 [e80264b3], DB [ebc715e6, e80264b3]
track  6: rip accurate     (max confidence    200) v1 [7d0e8bd8], v2 [7ee4aea1], DB [7d0e8bd8, 7ee4aea1]
track  7: rip accurate     (max confidence    200) v1 [1ebdee8b], v2 [ce255671], DB [1ebdee8b, ce255671]
track  8: rip accurate     (max confidence    200) v1 [7c54d45a], v2 [436b5b96], DB [7c54d45a, 436b5b96]
track  9: rip accurate     (max confidence    200) v1 [17d8439c], v2 [2799b73f], DB [17d8439c, 2799b73f]
track 10: rip accurate     (max confidence    200) v1 [d6a501b3], v2 [eb433c2e], DB [d6a501b3, eb433c2e]
track 11: rip accurate     (max confidence    200) v1 [81b6a780], v2 [0a09e206], DB [81b6a780, 0a09e206]
track 12: rip accurate     (max confidence    200) v1 [e768a4cf], v2 [7b2c21bc], DB [e768a4cf, 7b2c21bc]
track 13: rip accurate     (max confidence    200) v1 [25fab0d9], v2 [058ee8c7], DB [25fab0d9, 058ee8c7]
track 14: rip accurate     (max confidence    200) v1 [d92fd059], v2 [11f2a1af], DB [d92fd059, 11f2a1af]


####################################

Finally the resulting directory structure:

####################################

klaus@nuc:/tmp/newrip/Norah Jones-Come Away With Me-2002$ ll

total 271664

-rw------- 1 klaus klaus 19656091 Jul 26 12:51 "01-Norah Jones-Come Away With Me-Don't Know Why.flac"
-rw------- 1 klaus klaus 14536156 Jul 26 12:52 '02-Norah Jones-Come Away With Me-Seven Years.flac'
-rw------- 1 klaus klaus 19441995 Jul 26 12:53 '03-Norah Jones-Come Away With Me-Cold Cold Heart.flac'
-rw------- 1 klaus klaus 20656186 Jul 26 12:55 "04-Norah Jones-Come Away With Me-Feelin' the Same Way.flac"
-rw------- 1 klaus klaus 21162886 Jul 26 12:56 '05-Norah Jones-Come Away With Me-Come Away With Me.flac'
-rw------- 1 klaus klaus 25041230 Jul 26 12:57 '06-Norah Jones-Come Away With Me-Shoot the Moon.flac'
-rw------- 1 klaus klaus 16465039 Jul 26 12:58 '07-Norah Jones-Come Away With Me-Turn Me On.flac'
-rw------- 1 klaus klaus 19528817 Jul 26 12:59 '08-Norah Jones-Come Away With Me-Lonestar.flac'
-rw------- 1 klaus klaus 26935609 Jul 26 13:00 "09-Norah Jones-Come Away With Me-I've Got to See You Again.flac"
-rw------- 1 klaus klaus 15926445 Jul 26 13:01 '10-Norah Jones-Come Away With Me-Painter Song.flac'
-rw------- 1 klaus klaus 19358391 Jul 26 13:02 '11-Norah Jones-Come Away With Me-One Flight Down.flac'
-rw------- 1 klaus klaus 27185099 Jul 26 13:03 '12-Norah Jones-Come Away With Me-Nightingale.flac'
-rw------- 1 klaus klaus 16185706 Jul 26 13:03 '13-Norah Jones-Come Away With Me-The Long Day Is Over.flac'
-rw------- 1 klaus klaus 16060035 Jul 26 13:04 '14-Norah Jones-Come Away With Me-The Nearness of You.flac'
-rw-r--r-- 1 klaus klaus     1690 Jul 26 13:04 'Norah Jones-Come Away With Me-2002.m3u'
-rw-r--r-- 1 klaus klaus     2003 Jul 26 13:04 'Norah Jones-Come Away With Me-2002.cue'
-rw-r--r-- 1 klaus klaus    10199 Jul 26 13:05 'Norah Jones-Come Away With Me-2002.log'


Annex 3

Drive offset query 

Since my Lite-ON drive has an offset of 6 the offset query function works fine. 
Because "6" is the first offset on the list of numerous offsets that's being tried on the drive. 
Any drive. 
If you'd have an offset of e.g. 696. It'll take forever. I promise! And it'll put a hell of a load on drive.
As I said earlier. It's better to search for the model id of your drive in the ACR database manually.

For this test I used the Norah Jones keydisk again.

#####################################################################
klaus@nuc:~$ whipper offset find
Checking device /dev/sr0
eject: CD-ROM tray close command failed: Input/output error
Trying read offset 6 ...
Offset of device is likely 6, confirming ...
                                         
Read offset ocdf device is: 6.

Adding read offset to configuration file.
#############################################################################

During the process the entire CD gets ripped to confirm that the selected offset is correct! Better be patient!

Once succesfully finished whipper adds the drive parameters to the config file. It'l generate it if it doesn't exist
under ~/.config/whipper

That's how the config file will look like after the execise:


######################################################################

klaus@nuc:~$ cat .config/whipper/whipper.conf 
[drive:Slimtype%3AeBAU108%20%206%20L%20%20%20%20%3APL0M]
vendor = Slimtype
model = eBAU108  6 L
release = PL0M
read_offset = 6


###############################################################

Note: As you see the drive is not recognized as a LiteOn device. What matters is the model id.
Unfortunately that's the way it's also listed in the Accurate Rip drive database. 
You won't find this specifc drive under "LiteOn" devices in that ACR list.





No comments:

Post a Comment