Wednesday, December 09, 2015

How To Boot an NVME SSD from a Legacy BIOS (non UEFI) w/ Intel 750 Series SSD

This tutorial is about LEGACY BIOSes.

This is something i've been wanting to try for a very long time, since I have an older X58 motherboard, which is still my main system, and I'm using a Core i7 950 OC'ed to 3.8GHz.

EVERYONE said it couldn't be done. That is no longer true, because I have proven it myself!
It involves a program/bootloader called Tianocore, which has mostly been used for as part of "Clover" to boot Hackintoshes in EFI mode. But it will serve us so we can boot Windows in EFI mode, so the NVME drive is able to be detected at the boot stage, prior to Windows loading.

Background Info (scroll down to see solution, if you are lazy)

Only a UEFI compatible with NVME (something like UEFI version 2.3.1 and later) along with a "efi driver"/module will allow the computer to POST->BOOT->OS from an NVME drive.
There are other websites explaining how to mod your UEFI if you need to add this support into an older UEFI.

  • The Legacy BIOS sees the card as a generic undefined "SCSI Card" in the boot devices list, but this will not work.
  • The Windows 10 OS (what I used) - going back to Windows 8.1 (not 8) - come included with a "Standard NVME Controller driver" which lets Windows use the drive as a data drive ( D:\ for example).
  • The Windows 10 Installer sees the drive immediately but refuses to let you install to it because of a double-checking routine which queries the BIOS to make sure your system WOULD be bootable, so it doesn't waste your time installing to something impossible to boot from. (This is quite annoying in this case, but we can work around it by any number of ways, Cloning is one... Unrelatedly, also of note, this BIOS double-check also applies to Hot-plugging a SATA drive (for me anyway, as my BIOS has no options for supporting Hot Plug detection, even though the SATA Controllers do.)
  • The Intel 750 Series PCI-e NVME SSD has an Option ROM flash chip on the card. Dumping the ROM from linux gets us:

    Image 1:

      PCI Expansion ROM Header:

        Signature: 0x55aa (Ok)

        CPU unique data: 0x2b 0x00 0xf1 0x0e 0x00 0x00 0x0b 0x00

                         0x64 0x86 0x01 0x00 0x00 0x00 0x00 0x00

        Pointer to PCI Data Structure: 0x001c

    

      PCI Data Structure:

        Signature: 0x50434952 'PCIR' (Ok)

        Vendor ID: 0x8086

        Device ID: 0x0953
        Vital Product Data:  0x0000
        PCI Data Structure Length: 0x001c (28 bytes)
        PCI Data Structure Revision: 0x03
        Class Code: 0x000000 (Legacy Device)
        Image Length: 0x002b blocks (22016 bytes)
        Revision Level of Code/Data: 0x0000
        Code Type: 0x03 (Intel EFI (unofficial))
        Last-Image Flag: 0x80 (last image in rom)
        Reserved: 0x0000
      Parsing of platform specific data not available for this image


Some things to note: Class Code: Legacy Device, well, yes it is a Legacy device, you can use it just fine as long as you aren't booting to it. Code Type: Intel EFI. and then the last line is the most telling (especially when you compare it to a ROMdump of something like a network card) - reveals that this boot ROM is only applicable to EFI platforms (not BIOS). Therefore we cannot just mod/add the vendor device ID 8086_0953 to the BIOS tables to get it to boot.
Theoretically, if we could find a legacy BIOS based option ROM for this card, we could re-flash the card with that as a solution however no such thing exists and likely never will.

Some History on Windows, and the Boot Process


I thought by using a 2nd visible SATA HDD to initiate the MBR boot instruction, then calling BOOTMGR to obtain the List of Windows Installations (where you push F8 for safe mode etc) would be enough, but this also falls dead in its tracks.... The process goes like this: POST->BIOS->MBR->BOOTMGR->winload.exe->failure. Winload.exe is what loads NTOSKRNL.exe and the HAL.dll and all of the drivers classified as "Boot" (like hard drive controllers). The problem is that it FAILS to reach the point of loading any drivers. I thought I could trick the BCD Editor with tricky manual editing (bcdedit.exe) but no amount of editing will allow winload.exe to see an NVME drive as a valid path (even though Windows has the driver, its just too early in the process, and this native code is relying heavily on the BIOS).

This is an entry for a default regular MBR/BIOS install of Windows:

Windows Boot Loader
-------------------
identifier              {current}
device                  partition=C:
wath                    \Windows\system32\winload.exe
description             Windows 10
locale                  en-us
inherit                 {bootloadersettings}
allowedinmemorysettings 0x15000075
osdevice                partition=C:
systemroot              \Windows
resumeobject            {fdd8d97c-9d4e-11e5-9c8b-bcaec519e8ac}
nx                      OptIn
bootmenupolicy          Standard 

You can see "device" = C: and path = winload.exe. Then you also see "osdevice" = C: and systemroot = \Windows. Changing one or both to D: or anything else (many hours of trial and error).... does not work.
BCD configuration only accepts drive letters, or paths in the form \Device\HarddiskVolume2, which is IMMEDIATELY converted into a drive letter (if mounted). But actually in the background, whats happening is that the Volume Serial Number/partition signature (an 8 or 16 digit hex number on the partition's logical sector 0) is being located and stored. This is why many times if you clone a drive, because some cloning programs decide its best to change the signature, and then do not correctly update the boot config to match. It's not just "Boot from Drive C:" or "boot from partition 1".

No amount of trickery with bootloaders, chainloaders, GRUB, or modifying windows's settings/ registry will help with this.


SOLUTION: 10 Steps to Take to Really Make it work

0. Start off with your NVME drive initialized as MBR, not GPT. (I have not tested GPT.) 
1. Create a small FAT32 partition like ~100MB on the NVME drive as partition 1. (not under 32MB or FAT32 won't work - i think windows has some larger-than32mb but less-than100mb abnormal minimum limit before it refuses to format, just stick to 100MB).  Name it something like "EFI BOOT" for good measure. Assign it a drive letter to be able to access it. Make it Active.
          C:\> diskpart.exe
     DISKPART> list disk
     DISKPART> select disk 4
     DISKPART> clean      (start fresh: removes any partitions/the MBR. skip if needed)
     DISKPART> convert mbr (starting fresh: initializes disk as MBR. skip if needed)
     DISKPART> create partition primary SIZE=100
     DISKPART> list partition
     DISKPART> select partition 1
     DISKPART> format fs=fat32 label=efi_boot
     DISKPART> assign letter=K
     DISKPART> active
     DISKPART> exit
2. Clone a partition already containing Windows to the available space, as partition 2. Since the installer is fail-tastic. Theoretically you can install a fresh version, but I did not do this, and you would need a UEFI copy of the windows installation media to launch the installer in UEFI mode.
Now you need to make sure the registry on the new cloned disk has C: pointing to the NVME disk's Disk Signature. Some cloning programs do this automatically.  HKLM\SYSTEM\MountedDevices\ holds this key. The best way to do this is to clone an OS that has seen the NVME drive that you just initialized. (since initializing resets the disk signature).  If you have an old image you want to re-use, export the Reg-key of the NVME drive's letter from the running OS, and save it so you can add it in the next step.
2a. Editing the registry ON the cloned drive: This requires manually loading the cloned drive's SYSTEM hive in a running OS.  Then load regedit.exe, click the HKLM key, click File|Load Hive..., type any name. Now navigate to  HKLM\SYSTEM\MountedDevices\ , Then you can delete the existing entry for C:, find the letter that the NVME drive's windows partition existed as, and  rename that key/letter so its the new C:. (or import the reg-key you exported) Then unload the reg-hive (it auto-saves). 
3. Create the boot files on the FAT32 partition to boot the newly cloned NTFS windows partition. Get an admin command prompt ready and type:
bcdboot L:\Windows /s K: /h UEFI
where L:\Windows is the NVME drive's windows directory and K: is the FAT32 partition on the NVME drive. (dont worry that L: is not called C:, this part does not obey the \MountedDevices list you just edited)
4. Your'e going to want some kind of MBR master boot sector on this drive I think (I forget how I did this. Maybe bcdboot puts one.).
5. Download Tianocore (the entire zip file) from here https://gitlab.com/tianocore_uefi_duet_builds/tianocore_uefi_duet_installer/tree/master
This is an open source EFI "emulator" . Extract it somewhere making sure the pathname has no spaces in it.
6. They intend for you to create a USB Flash drive to boot with, which you should do first, to test it. (you can worry about creating something more permanent like a 2nd internal disk later - see bottom of this page). Run:
          CreateUSB.cmd M:
where M: is the letter of your USB flash drive. Do NOT put this on the NVME drive, that would be dumb. You want to trigger the NVME boot partition from something thats ACTUALLY bootable, in this case a USB drive.
6a. Plug and replug it. If it shows up as Non-Formatted, Thats not right. I had issues with this. Doing Diskpart.exe: clean , followed by, convert mbr , will start fresh. to clear out any latent crap in the MBR or just try a new flash drive.
6b. USB Creation Step 2, run:
          CreateUSB.cmd M: UDK_X64
6c. Make the partition active. (diskpart or diskmgmt.msc)
7. Test the USB Flash drive. Boot off it. Make sure you see "Tianocore" logo, and a little menu hopefully. You can poke around in here, but really no configuration is needed.
8. The NVME drive should be triggered and show a Windows Logo or blue windows screen now. Windows will not load fully however. (INACCESSIBLE_BOOT_DEVICE) You have to press F8 and boot into Safe Mode First. I'm not exactly sure why that is necessary, but it is. (I've repeated the process twice to be sure.) I feel like safe mode resets something in the registry that I dont know about.
9. Reboot, off the USB flash drive, and now Tianocore should boot, and then Windows should boot into normal mode. Perfect!
10. DONE!

Now you probably want to know how to make it so you don't have a USB stick hanging out of your system for the rest of its natural life. you are going to need a SECOND FAT32 partition on a SATA disk, and it is going to have to be Partition 1. I have not figured out a way to allow this partition to be at the end of the disk to save effort of resizing the partition. The best partition management tool i've found is "MiniTool Partition Wizard Home Edition 9" available for free, full featured. They will resize very fast, safely, and in windows without rebooting, even resizing to make space in the FRONT. Once you have that small (again like 100MB FAT32) partition on a bootable SATA disk, the hard part comes. The damn CreateUSB.cmd script relies on some programs called BootSectImage.exe and GenBootSect.exe and they modify the MBR and the PBR of a USB flash drive. Examining the source code of these files reveals that they should be compatible with IDE disks, but yet my testing reveals they do not work on SATA HARD DRIVES at least in Windows. You should try first running them from a NATIVE DOS prompt to see if they are treated as Legacy IDE in DOS and it allows to work. If not, well, you have to cheat. Run the following procedures,editing the first line to point to the path that you extracted the Tianocore Zip file to. Pseudo commands and actual commands are mixed because I have not automated this as of yet to set this up.


<STEP 0: SET ENV VARIABLES>
set UEFI_DUET=E:\750_Project\tiano\
set BOOTSECTOR_BIN_DIR=%UEFI_DUET%\BootSector

set WIN_BIN_DIR=%UEFI_DUET%\Windows_Binaries
cd %UEFI_DUET%

mkdir MY_SECTORS

<STEP 1: Patch the PBR>
#back up the desired SATA HDD FAT32 
partition's existing boot sector (sector 2048) manually to a file called Fat32-2048.bin with secinspect.exe or other tool. #
# https://www.microsoft.com/en-us/download/details.aspx?id=19470 #

copy %BOOTSECTOR_BIN_DIR%\bs32.com MY_SECTORS\newbs32.bin


SECINSPECT.EXE -backup Q: MY_SECTORS\Fat32-2048.bin 2048 1
%WIN_BIN_DIR%\BOOTSECTIMAGE.EXE --verbose -g MY_SECTORS\Fat32-2048.bin MY_SECTORS\newBS32.bin -f
# now write this newbs32.bin back to 2048 and 2054(backup) #
SECINSPECT.EXE -restore Q: MY_SECTORS\newBS32.bin 2048 CONFIRM
SECINSPECT.EXE -restore Q: MY_SECTORS\newBS32.bin 2054 CONFIRM

<STEP 2: Patch the MBR>
The MBR at sector 0 contains two sections, from hex addresses 0-1B8, the boot code. From that point on, exists the partition table which you do not want to mess with. You can either Hex-Edit the MBR and copy and paste the code from the file Mbr.com, (which is what I did as a learning process) or do a similar procedure:

SECINSPECT.EXE -backup Q: MY_SECTORS\MBR-sec0.bin 0 1
%WIN_BIN_DIR%\GENBOOTSECTOR.EXE  --verbose --mbr -o MY_SECTORS\MBR-sec0.bin -i %BOOTSECTOR_BIN_DIR%\Mbr.com 
SECINSPECT.EXE -restore Q: MY_SECTORS\MBR-sec0.bin 0 CONFIRM

You now have done step 1 and 2 . Proceed to Step 3 (only copies files):
          CreateUSB.cmd Q: UDK_X64

This should be all thats required to get the equivalent of the USB disk onto your HDD. Now everything is done. Enjoy. Simple huh :) Hope you could understand this all, I put a lot of time into it. email me @ mrlithium NOT@AT comcast.net if you need help.