Raspberry Pi 5 - M.2 NVME SSD

Why M.2 NVME SSD on a RPi5???  

We might intend to create an even better audio streaming giant !?!?  ;)

I am always looking for increasing efficiency on the system. Efficiency? An M.2-NVME SSD attached via PCIe - it can't get any faster - except you use 100% RAM drives.  

The usual USB route for SSDs adds another layer and a Sata/USB or M.2/USB conversion on top. There's a good chance that we can increase the efficiency of the overall system by going the M.2 route. Beside that the USB DAC remains the only device attached to RPi USB. And perhaps we can save some $ on USB filter gadgets. Yep. Plenty of good reasons.

The Raspberry Pi 5 offers a M.2 port right next to the Wifi antenna. It allows to attach a M.2 NVME SSD HAT. You need a carrier board for your SSD to go that route. I currently use a board from Pineberry and a 2230 SSD. There are more of these kind of boards out there. E.g. from Pimoroni (on order). Most of these boards are hard to come by in this early phase and many of them are WiP (work-in-progress). Meaning. You'll see updates popping in.   

The RPi 5 allocates a single PCIe lane to that I/O port. The foundation guarantees PCIe Gen2 5Gbit/s over that port. A not officially supported and not certified Gen3 mode allows even 10Gbit/s. The foundation says operation might get unstable in Gen3 mode. I'll of course give it a try anyhow. ;)

The RPi5 requires solid cooling. Therefore I wouldn't like to attach the carrier board on top (HAT) of or underneath the RPi. The situation: PI heat is also spreaded towards the bottom - the PI circuit board also acts as a cooling device and not to forget the SSD also generates quite some heat. If you put all that underneath the RPI you might create a jammed heat bubble at the bottom.

These considerations led to a setup as shown below:

As you can see I put the carrier board on stands. A slight problem with my case of choice from Geeekom is that I have to dismount the case for connecting the cable and the FFC cable gets bend quite sharply. That's not good, not good at all. Sharp bends are causing impedance issues on these high-speed links. The foundation specifies a length of 50mm max for the cable! And an impedance of 90R. Unfortunately I haven't seen any HAT manufacturer offering FFC cables as spare parts. These can break easily. Please ask for it - if you order a HAT. The more people ask, the earlier we can order them. 

CAUTION: You don't want to put these flimsy onboard FFC connectors under stress!

And... I already changed my setup to accomplish exactly a lower stressed link. I couldn't live with it

Introducing: The Notch

I took out my Dremel and cut out a notch from my GeeeKom case. Now the cable floats freely.
And I can mount and dismount the cable easily without dismounting the heatsink. Keep in mind. Dismounting the heatsink might lower the cooling efficiency! These cooling pads work best on
the first application. Any more mounts/dismounts might lower the efficiency of these pads! 

I further mounted the whole stuff on a wooden base and put some space between the bottom fins and the base. 

Don't let that flimsy FFC (flexible flat cable) wriggle around too much. You can easily stress it out.

The good part is that up to 5W of power can be sourced over the ribbon cable. There's no need for another external power supply. You'd need to know the power consumption of your SSD to see if it works.

The bad part is that you can not choose between external OR internal powering with the Pineberry board. If you attach power through the GPIO header you'd face two concurrent power paths. 

CAUTION: If you just attach power to the HAT in above setting (I did - because Pineberry told me the board would recognize the power source - cable or header) - you'll power the PI through the FFC cable!
You better avoid that to happen!

Now. There are two more aspects to consider. First. Cooling of SSDs is pretty much standard nowadays.
Pretty much every external case you buy offers SSD cooling pads and uses the case as cooling device. None of the M.2 HATs I've seen offers a cooling option. Perhaps the manufacturers should put a thought on that. Second. This open cabling and open HAT board is all but a great solution when it comes to EMI/RFI and direct access. A well covered USB-M.2 case and cable would look like a much preferred solution to me here.

What I also don't like on the Pineberry board is that you can not turn off the LEDs. The guy from Pineberry simply avoided to answer my related question. 

Let's think a bit about the handling. Before you start the mounting exercise you might first ask yourself how to get the OS on the SSD!!!
And how to run the backups afterwards. My Waveshare-IO board that I used in the past, allowed me to boot the CM4 in OTG gadget mode from the PC. This way it was easy to install and backup anything from the M.2-SSD without unmounting the SSD. It's different here.

I see two options.

1. You dismount the SSD, attach it to a M.2-USB adapter and run the installation or backup
2. You boot from an USB or SD drive with a second image and transfer data back and forth from that drive to the M.2 SSD

Yes. It's gonna be annoying. I'm just saying...

Now. To be able to boot the M.2 SSD some minor configurations need to be done. The official related documentation can be found over at the RPI Docs pages. I'd like to quickly summarize it.

1. Make sure you've got the latest eeprom firmware installed.

2. Make sure you've got the latest system update incl. latest kernel installed.

3. We then need to edit the /boot/firmware/config.txt file 

In /boot/firmware/config.txt we need to add

## enable the external PCIe Gen 2.0 output

dtparam=pciex1

## force Gen 3.0 speeds (optional) - remove the # in the following line

#dtparam=pciex1_gen=3

4. The rpi-eeprom needs to be edited. Following two new lines are required

BOOT_ORDER=0xf146 

PCIE_PROBE=1 

The RPi will now try to boot the M.2 (6) first, then USB (4) and then the SD (1).

If you  want to use option 2. of my "how to get the data on and off the SSD" topic, meaning, you want o boot from USB to backup the NVMe drive you need to do some configurations.

1. The rpi-eeprom needs to be edited. 

BOOT_ORDER=0xf164 

 

Yep. You need to remove (for now) the PCIE_PROBE=1 entry.  Otherwise the NVME will not be shown after USB boot.  The RPi then first tries to boot the USB. 

To edit the eeprom accordingly you simply run 

sudo -E rpi-eeprom-config --edit

sudo reboot

NOTE: You can't use an USB drive installation that's been the exact copy of the NVME drive.
Both drives will than have the same UUIDs - Unique Identifiers - configured, which will cause confusion on the system during the boot process!

 

Once you're finished with the backup or restore you need to edit the config once more.

Conclusion

The whole thing IMO still is WiP (Work-in-Progress). HATs are updated as I am writing these lines (I just learned that from Pineberry), and the RPi kernel folks running this or that regression and update on the kernel and firmware part. 

Having a naked board hanging around is not such a great setting. 

The whole backup and restore process is a bit flaky and annoying.

Attaching an external power supply almost smoked my setup. It's IMO still worth to consider external powering for offloading the PI rails. 

Bottom line. I'd closely follow the developments before hitting the "buy" button. If you can wait a little - I'd say from Q2-3/2024 onwards things might become more attractive and stable. (I know - manufacturers are watching.)

However. 

The - my - main goal - to increase the system efficiency and to offload the USB bus - was achieved.  I am quite happy with that result. 

BUT. All the effort and restrictions around the whole project - at this point in time - IMO is not worth the hassle.

Enjoy.