Wednesday, December 09, 2015

How To Boot an NVME SSD from a Legacy BIOS (non UEFI) w/ Intel 750 Series SSD

This tutorial is about LEGACY BIOSes.

This is something i've been wanting to try for a very long time, since I have an older X58 motherboard, which is still my main system, and I'm using a Core i7 950 OC'ed to 3.8GHz.

EVERYONE said it couldn't be done. That is no longer true, because I have proven it myself!
It involves a program/bootloader called Tianocore, which has mostly been used for as part of "Clover" to boot Hackintoshes in EFI mode. But it will serve us so we can boot Windows in EFI mode, so the NVME drive is able to be detected at the boot stage, prior to Windows loading.

Background Info (scroll down to see solution, if you are lazy)

Only a UEFI compatible with NVME (something like UEFI version 2.3.1 and later) along with a "efi driver"/module will allow the computer to POST->BOOT->OS from an NVME drive.
There are other websites explaining how to mod your UEFI if you need to add this support into an older UEFI.

  • The Legacy BIOS sees the card as a generic undefined "SCSI Card" in the boot devices list, but this will not work.
  • The Windows 10 OS (what I used) - going back to Windows 8.1 (not 8) - come included with a "Standard NVME Controller driver" which lets Windows use the drive as a data drive ( D:\ for example).
  • The Windows 10 Installer sees the drive immediately but refuses to let you install to it because of a double-checking routine which queries the BIOS to make sure your system WOULD be bootable, so it doesn't waste your time installing to something impossible to boot from. (This is quite annoying in this case, but we can work around it by any number of ways, Cloning is one... Unrelatedly, also of note, this BIOS double-check also applies to Hot-plugging a SATA drive (for me anyway, as my BIOS has no options for supporting Hot Plug detection, even though the SATA Controllers do.)
  • The Intel 750 Series PCI-e NVME SSD has an Option ROM flash chip on the card. Dumping the ROM from linux gets us:

    Image 1:

      PCI Expansion ROM Header:

        Signature: 0x55aa (Ok)

        CPU unique data: 0x2b 0x00 0xf1 0x0e 0x00 0x00 0x0b 0x00

                         0x64 0x86 0x01 0x00 0x00 0x00 0x00 0x00

        Pointer to PCI Data Structure: 0x001c

    

      PCI Data Structure:

        Signature: 0x50434952 'PCIR' (Ok)

        Vendor ID: 0x8086

        Device ID: 0x0953
        Vital Product Data:  0x0000
        PCI Data Structure Length: 0x001c (28 bytes)
        PCI Data Structure Revision: 0x03
        Class Code: 0x000000 (Legacy Device)
        Image Length: 0x002b blocks (22016 bytes)
        Revision Level of Code/Data: 0x0000
        Code Type: 0x03 (Intel EFI (unofficial))
        Last-Image Flag: 0x80 (last image in rom)
        Reserved: 0x0000
      Parsing of platform specific data not available for this image


Some things to note: Class Code: Legacy Device, well, yes it is a Legacy device, you can use it just fine as long as you aren't booting to it. Code Type: Intel EFI. and then the last line is the most telling (especially when you compare it to a ROMdump of something like a network card) - reveals that this boot ROM is only applicable to EFI platforms (not BIOS). Therefore we cannot just mod/add the vendor device ID 8086_0953 to the BIOS tables to get it to boot.
Theoretically, if we could find a legacy BIOS based option ROM for this card, we could re-flash the card with that as a solution however no such thing exists and likely never will.

Some History on Windows, and the Boot Process


I thought by using a 2nd visible SATA HDD to initiate the MBR boot instruction, then calling BOOTMGR to obtain the List of Windows Installations (where you push F8 for safe mode etc) would be enough, but this also falls dead in its tracks.... The process goes like this: POST->BIOS->MBR->BOOTMGR->winload.exe->failure. Winload.exe is what loads NTOSKRNL.exe and the HAL.dll and all of the drivers classified as "Boot" (like hard drive controllers). The problem is that it FAILS to reach the point of loading any drivers. I thought I could trick the BCD Editor with tricky manual editing (bcdedit.exe) but no amount of editing will allow winload.exe to see an NVME drive as a valid path (even though Windows has the driver, its just too early in the process, and this native code is relying heavily on the BIOS).

This is an entry for a default regular MBR/BIOS install of Windows:

Windows Boot Loader
-------------------
identifier              {current}
device                  partition=C:
wath                    \Windows\system32\winload.exe
description             Windows 10
locale                  en-us
inherit                 {bootloadersettings}
allowedinmemorysettings 0x15000075
osdevice                partition=C:
systemroot              \Windows
resumeobject            {fdd8d97c-9d4e-11e5-9c8b-bcaec519e8ac}
nx                      OptIn
bootmenupolicy          Standard 

You can see "device" = C: and path = winload.exe. Then you also see "osdevice" = C: and systemroot = \Windows. Changing one or both to D: or anything else (many hours of trial and error).... does not work.
BCD configuration only accepts drive letters, or paths in the form \Device\HarddiskVolume2, which is IMMEDIATELY converted into a drive letter (if mounted). But actually in the background, whats happening is that the Volume Serial Number/partition signature (an 8 or 16 digit hex number on the partition's logical sector 0) is being located and stored. This is why many times if you clone a drive, because some cloning programs decide its best to change the signature, and then do not correctly update the boot config to match. It's not just "Boot from Drive C:" or "boot from partition 1".

No amount of trickery with bootloaders, chainloaders, GRUB, or modifying windows's settings/ registry will help with this.


SOLUTION: 10 Steps to Take to Really Make it work

0. Start off with your NVME drive initialized as MBR, not GPT. (I have not tested GPT.) 
1. Create a small FAT32 partition like ~100MB on the NVME drive as partition 1. (not under 32MB or FAT32 won't work - i think windows has some larger-than32mb but less-than100mb abnormal minimum limit before it refuses to format, just stick to 100MB).  Name it something like "EFI BOOT" for good measure. Assign it a drive letter to be able to access it. Make it Active.
          C:\> diskpart.exe
     DISKPART> list disk
     DISKPART> select disk 4
     DISKPART> clean      (start fresh: removes any partitions/the MBR. skip if needed)
     DISKPART> convert mbr (starting fresh: initializes disk as MBR. skip if needed)
     DISKPART> create partition primary SIZE=100
     DISKPART> list partition
     DISKPART> select partition 1
     DISKPART> format fs=fat32 label=efi_boot
     DISKPART> assign letter=K
     DISKPART> active
     DISKPART> exit
2. Clone a partition already containing Windows to the available space, as partition 2. Since the installer is fail-tastic. Theoretically you can install a fresh version, but I did not do this, and you would need a UEFI copy of the windows installation media to launch the installer in UEFI mode.
Now you need to make sure the registry on the new cloned disk has C: pointing to the NVME disk's Disk Signature. Some cloning programs do this automatically.  HKLM\SYSTEM\MountedDevices\ holds this key. The best way to do this is to clone an OS that has seen the NVME drive that you just initialized. (since initializing resets the disk signature).  If you have an old image you want to re-use, export the Reg-key of the NVME drive's letter from the running OS, and save it so you can add it in the next step.
2a. Editing the registry ON the cloned drive: This requires manually loading the cloned drive's SYSTEM hive in a running OS.  Then load regedit.exe, click the HKLM key, click File|Load Hive..., type any name. Now navigate to  HKLM\SYSTEM\MountedDevices\ , Then you can delete the existing entry for C:, find the letter that the NVME drive's windows partition existed as, and  rename that key/letter so its the new C:. (or import the reg-key you exported) Then unload the reg-hive (it auto-saves). 
3. Create the boot files on the FAT32 partition to boot the newly cloned NTFS windows partition. Get an admin command prompt ready and type:
bcdboot L:\Windows /s K: /h UEFI
where L:\Windows is the NVME drive's windows directory and K: is the FAT32 partition on the NVME drive. (dont worry that L: is not called C:, this part does not obey the \MountedDevices list you just edited)
4. Your'e going to want some kind of MBR master boot sector on this drive I think (I forget how I did this. Maybe bcdboot puts one.).
5. Download Tianocore (the entire zip file) from here https://gitlab.com/tianocore_uefi_duet_builds/tianocore_uefi_duet_installer/tree/master
This is an open source EFI "emulator" . Extract it somewhere making sure the pathname has no spaces in it.
6. They intend for you to create a USB Flash drive to boot with, which you should do first, to test it. (you can worry about creating something more permanent like a 2nd internal disk later - see bottom of this page). Run:
          CreateUSB.cmd M:
where M: is the letter of your USB flash drive. Do NOT put this on the NVME drive, that would be dumb. You want to trigger the NVME boot partition from something thats ACTUALLY bootable, in this case a USB drive.
6a. Plug and replug it. If it shows up as Non-Formatted, Thats not right. I had issues with this. Doing Diskpart.exe: clean , followed by, convert mbr , will start fresh. to clear out any latent crap in the MBR or just try a new flash drive.
6b. USB Creation Step 2, run:
          CreateUSB.cmd M: UDK_X64
6c. Make the partition active. (diskpart or diskmgmt.msc)
7. Test the USB Flash drive. Boot off it. Make sure you see "Tianocore" logo, and a little menu hopefully. You can poke around in here, but really no configuration is needed.
8. The NVME drive should be triggered and show a Windows Logo or blue windows screen now. Windows will not load fully however. (INACCESSIBLE_BOOT_DEVICE) You have to press F8 and boot into Safe Mode First. I'm not exactly sure why that is necessary, but it is. (I've repeated the process twice to be sure.) I feel like safe mode resets something in the registry that I dont know about.
9. Reboot, off the USB flash drive, and now Tianocore should boot, and then Windows should boot into normal mode. Perfect!
10. DONE!

Now you probably want to know how to make it so you don't have a USB stick hanging out of your system for the rest of its natural life. you are going to need a SECOND FAT32 partition on a SATA disk, and it is going to have to be Partition 1. I have not figured out a way to allow this partition to be at the end of the disk to save effort of resizing the partition. The best partition management tool i've found is "MiniTool Partition Wizard Home Edition 9" available for free, full featured. They will resize very fast, safely, and in windows without rebooting, even resizing to make space in the FRONT. Once you have that small (again like 100MB FAT32) partition on a bootable SATA disk, the hard part comes. The damn CreateUSB.cmd script relies on some programs called BootSectImage.exe and GenBootSect.exe and they modify the MBR and the PBR of a USB flash drive. Examining the source code of these files reveals that they should be compatible with IDE disks, but yet my testing reveals they do not work on SATA HARD DRIVES at least in Windows. You should try first running them from a NATIVE DOS prompt to see if they are treated as Legacy IDE in DOS and it allows to work. If not, well, you have to cheat. Run the following procedures,editing the first line to point to the path that you extracted the Tianocore Zip file to. Pseudo commands and actual commands are mixed because I have not automated this as of yet to set this up.


<STEP 0: SET ENV VARIABLES>
set UEFI_DUET=E:\750_Project\tiano\
set BOOTSECTOR_BIN_DIR=%UEFI_DUET%\BootSector

set WIN_BIN_DIR=%UEFI_DUET%\Windows_Binaries
cd %UEFI_DUET%

mkdir MY_SECTORS

<STEP 1: Patch the PBR>
#back up the desired SATA HDD FAT32 
partition's existing boot sector (sector 2048) manually to a file called Fat32-2048.bin with secinspect.exe or other tool. #
# https://www.microsoft.com/en-us/download/details.aspx?id=19470 #

copy %BOOTSECTOR_BIN_DIR%\bs32.com MY_SECTORS\newbs32.bin


SECINSPECT.EXE -backup Q: MY_SECTORS\Fat32-2048.bin 2048 1
%WIN_BIN_DIR%\BOOTSECTIMAGE.EXE --verbose -g MY_SECTORS\Fat32-2048.bin MY_SECTORS\newBS32.bin -f
# now write this newbs32.bin back to 2048 and 2054(backup) #
SECINSPECT.EXE -restore Q: MY_SECTORS\newBS32.bin 2048 CONFIRM
SECINSPECT.EXE -restore Q: MY_SECTORS\newBS32.bin 2054 CONFIRM

<STEP 2: Patch the MBR>
The MBR at sector 0 contains two sections, from hex addresses 0-1B8, the boot code. From that point on, exists the partition table which you do not want to mess with. You can either Hex-Edit the MBR and copy and paste the code from the file Mbr.com, (which is what I did as a learning process) or do a similar procedure:

SECINSPECT.EXE -backup Q: MY_SECTORS\MBR-sec0.bin 0 1
%WIN_BIN_DIR%\GENBOOTSECTOR.EXE  --verbose --mbr -o MY_SECTORS\MBR-sec0.bin -i %BOOTSECTOR_BIN_DIR%\Mbr.com 
SECINSPECT.EXE -restore Q: MY_SECTORS\MBR-sec0.bin 0 CONFIRM

You now have done step 1 and 2 . Proceed to Step 3 (only copies files):
          CreateUSB.cmd Q: UDK_X64

This should be all thats required to get the equivalent of the USB disk onto your HDD. Now everything is done. Enjoy. Simple huh :) Hope you could understand this all, I put a lot of time into it. email me @ mrlithium NOT@AT comcast.net if you need help.

13 comments:

Ian Johns said...

Excellent work, much appreciated.

I have X58 motherboard, using Wind 10 and this gotten me really close but there are a few issues.

Can you clarify the version of DEUT you are using? The one linked does not support my 750 SSD (missing a driver?)

I had planned to use Clover and that does see the SSD and I can either install via it or clone over.

However I always get inaccessible boot device, even doing F8 (which needs metro boot turned off to get to press f8:))

Maybe this is clover related so hoping to try the version of Tianocore/Duet you have?

Thanks!

Kevin Hull said...

It works fine for me with an Intel 750, there are a few clerical errors though:

bcdboot L:\Windows /s K: /h UEFI
should be
bcdboot L:\Windows /s K: /f UEFI

Also the USB drive needs to be formatted with the "HP USB storage format tool" prior to the Tianocore process. This puts a true MBR on the USB drive. If you are getting inaccessible boot device BSOD then you probably did not edit the registry properly or your cloning software doesn't do it.

I modified this a bit for my needs so I could keep the original legacy boot sector in case I need to go back, but also use UEFI. This was my process:

- Paragon OS to SSD from my old 850 evo to the intel 750 from inside the existing windows installation.
- Create 100MB FAT32 partition between the system reserved partition and the windows partition
- Mark system reserved partition inactive
- Follow steps as indicated by MrLithium

Note: Paragon OS to SSD automatically adjusts the registry to the new drive, so you don't have to mess around with it.

robin lawrie said...

i have an odd and possibly related problem.. so far ive not been able to find much useful help anywhere.
i bought a samsung 950 pro 512gb and a pcie adapter card to use as a fast storage drive.
my current system is a rampage gene iv (x79) with windows 7 x64 installed on a raid 0 array (on intel controller) of 2x samsung 830 ssd's i also have 2x WD 2TB drives in raid1 mirror on the same controller for storage. 3930k OC, titan x gpu and 32 gb ram.

problem:
i plugged in the pcie drive, switched on, and my bios reset itself. then i had to load an old bios save and reapply all my settings.. very annoying.
reboot, aand: "please insert valid boot media"

checked and the bios had assigned the (clearly recognised) new pcie drive as the priimary boot option.

fixed that, disabling it in the list and setting my raid 0 boot drive as first option.
reboot: "please insert valid boot media"
check, right drive is chosen, retry , same problem.
remove pcie ssd, boots fine.

ive done a lot of researching about this, but most posts refer to booting FROM a pcie SSD.. i dont even want to do that. just want my machine to continue booting as normal, with an extra drive to use for (very big) stuff.

ive heard people suggest setting onboard controllers to ide mode.. obviously not an option for the intel controller as i need the raid arrays.
i tried disabling the asmedia controller entirely (and setting it to ide - its unused) didnt help.
i read that disabling CSM (?) might help (a related sounding issue) . but if i disable CSM and reboot, it just dumps me immediately back in bios.
i installed windows7 non-uefi (it wouldnt get through the install otherwise and i was advised to) so i guess thats why disabling CSM doesnt work..?

the advice ive got from an apparent expert is that these nvme drives *must* be used with uefi, and cannot be used with a legacy bios. your blog suggests this is only the case if you wish to boot..

Prafulla Giri said...

I can not explain how helpful this post is , I will try it on my 6 years old x58a workstation I recently ordered Intel 750 from newegg .i thought I just lost money on this drive but after reading this drive I have hope.

shaolin95 said...

So to clarify, this works when using a PCIe to SATA 3 adapter on a older x58 mobo like my Asus Rampage iii formular so that it can boot from it?
If so, will it work also with an M.2 drive?

Thanks!

PC Gaming Murah said...

it's good pc gaming kaskus

jasper said...

I realize this is an old thread, but if Kevin is reading this i have couple of questions for him I am hoping he can answer

kclem said...

I have been trying to get this to work with a Gigabyte GA-EX58-UD5P motherboard with an I7-920 in an LGA1366 socket using a
Lycom DT-120 M.2 PCI-e SSD to PCI-e Adapter Card to hold a
Samsung SM951 256GB M.2 PCI-e Gen3 8Gbps x 4 AHCI Solid State Drive (MZHPV256HDGL-00000).

Like another person I found the UDK_X64 did not load and give any tianocore logo but just a blank screen. The EDK_UEFI64 did load (no logo) but did not find anything to boot. It was possible to get a command line from the meu provided after the failed boot but I have little clue about what to do to investigate.

Please can anyone assist?

Note the SSD is seen by Windows 10 and I was able to build a Windows 10 OS on it and the Windows 10 boot loader does list it. It will not boot though, hence me trying the DUET loader.

davidm671 said...

I enjoyed reading your how to but found the steps a little confusing. You lost me at how you installed Tianocore to the second partition on the disk and how that in turn loads to OS.

Thanks

nicke85 said...

Is there some installation for Windows 10 which make this much easier?
eg. automatic configuration for mbr on nvme for fresh installation?

MRCS said...

Use NVMe SSD as a boot drive for Win7 x86 indirectly (AMD/Intel, UEFI/MBR)

MrLithium, great information. I ran into many of the same roadblocks for both x86 and x64.

For those of you with a similar problem on 32-bit Windows, I had previously started a topic over on the Seven Forums ( http://www.sevenforums.com/installation-setup/404203-use-nvme-ssd-boot-drive-win7-x86-sort.html ). By 'indirect' I mean the procedure uses a virtual machine. The post is not in its final form, but should be sufficient for most people.

Filip Dudan said...

https://audiocricket.com/2016/12/31/booting-samsung-sm961-on-asus-p6t-se-mainboard/
This post uses this concept and was very helpful in the process in making my SSD boot.

Thanks MrLithium!

Filip Dudan said...

UPDATE:

IF your MB allows you to create custom booting options, you can create partition on your sata3 drive, and start the DUET from there.
1. create 100 mb partiotion (primary)
2. format it to fat32
3. Assign letter (eg. B:)
4. run CreateUSB.cmd B: UDK_X64
5. Make BIOS start from that partition (shellx64_v1.efi)
Enjoy.