Heya! Hope you all are doing well. Me too. So, today I am gonna share something amazing and insightful. I have attended a meetup held by LinuxChix last Sunday. So after the talks, there held a chaos engineering session. Even though I hadn’t attended any session like this before, I was eager to experience these things myself. Because whenever I do something on my system, I am always very careful in case it doesn’t run into any system failures. So, this was a golden opportunity for me. In the session, my friend Neeraj and I were given a system with the error shown as:
error: file '/grub/i386-pc/normal.mod' not found.
grub rescue>
We have to resolve this error and get into the system safely. We were at the grub-rescue prompt. Basically grub-rescue prompt is somewhat more restricted as compared to grub prompt. This is because not all commands run in this shell. Only some selected commands work at the grub-rescue prompt. Very first command which we tried was “ls” command which worked and it showed the all the devices or partitions which were present in the system. So, the output was somewhat like:
grub rescue> ls (hd0) (hd0,msdos1) (hd0,msdos2) (hd0,msdos4) (hd0,msdos5) (hd0,msdos6)
These are the partitions which were present in the system. After that we run the “ls” command on each partition, which gives us some unexpected output. It tells if the file-system is known or not. Grub only recognizes the “ext2” file-system. So, we were left with only three partitions to work with which were (hd0,msdos2) (hd0,msdos4) (hd0,msdos6). Now you might be wondering what does this partition name means. So, if I would explain in short, it would be like
- (hd0,msdos1) = /dev/sda1
- (hd0,msdos2) = /dev/sda2
and so on. Now, in words (hd0,msdos1) will be the 1st partition in hard-drive 0 and (hd0,msdos2) means 2nd partition in hard-drive 0. Since the system had only single hard-drive and it’s been divided into partitions, so all partition contains (hd0). Partitions except the known file systems showed the error of “Unknown file system“.
Now, we were left with the 3 partitions to look into for the required file which is “normal.mod“. Before going any further, we should understand the need for this file. Basically, normal.mod is a module which grub loads at the boot time which provides the use of normal command. Using normal command we can display the GRUB menu. Since, the grub couldn’t locate the file in its usual location, it means either the file has been displaced or it could have been deleted somehow.
Case 1: ‘normal.mod’ is displaced from its default location
So, let’s consider the first case for now ie the file has been displaced and could be in some other partitions. Grub looks for this module in the /boot/grub/i386-pc/ by default(Here i386-pc is specific, it depends on your system architecture). For that we have to check the contents of each of the three partitions in order to ensure if the file is present there or not. We can do this using ls command.
grub rescue> ls (hd0,msdos1)/boot/grub/i386-pc/
We need to do this for every partition. If we get the required module in any of the partition. Let’s say we get it in the 1st partition. So, now we need to set the prefix variable to the partition which contains the required file. prefix variable is set to the location of grub directory at the time of grub installation. So, it is possible that grub couldn’t locate the normal.mod because the prefix variable is pointing to the wrong partition. So we can set the prefix variable using set command. Also, we need to set the root to the same partition as that of prefix . For eg
grub rescue> set root=(hd0,msdos1) grub rescue> set prefix=(hd0,msdos1)/boot/grub
Now, since we have set the correct path to root and prefix, we can use the normal module now. So for that we have to import the module. To import it, we would use insmod command.
grub rescue> insmod normal grub rescue> normal
Now, the grub menu would be shown and you can enter into the system safely.
Special case:
In this there can be situations like the normal.mod is not present in the exact location we are looking into. Like in our case, we are trying to locate it into “(hd0,msdos1)/grub/boot/i386-pc/”. But what if it is present in the location “(hd0,msdos1)/root/”, then pointing prefix to this location won’t be useful and also we can’t transfer it to its correct location. So, to resolve cases like this what we can do is use insmod command to import the normal module as:
grub rescue> insmod (hd0,msdos1)/root/normal.mod grub rescue> normal
Using this you would be able to use the normal module and get into the GRUB menu. But since it isn’t placed in its correct location, this means that there were some problem while installing grub. So, if you reboot the system, you would again be thrown into the grub rescue prompt and have to follow the same procedure. It would be better if you install the grub again using the following command after entering into the system terminal:
$ sudo grub-install /dev/sda
And then you are good to go.
Case 2: ‘normal.mod’ is not present
Well, the whole procedure was for the case when the required module is misplaced. But what if it isn’t present at all in any partition and somehow got deleted. To your curiousness, this was the case with us in chaos engineering session. We tried to find the module in each known partition but couldn’t locate it anywhere. So, now we have to do the work which normal module would’ve done if it would be present. Therefore we have to find the kernel files and boot the system manually.
To boot the system we would require the kernel file which would be something like “vimlinuz-linux” and the file “initramfs-linux.img“(Sometimes, you can find it as “initrd-linux.img”). The vimlinuz-linux is actually the kernel image which mounts the root partition on the hard-disk. The initramfs-linux.img loads the futhur required modules needed for booting of the system. So, very first thing we did was to locate these files in any of the known partitions. So, we looked for these files under the /boot directory using the same ls command as:
grub rescue> ls (hd0,msdos2)/boot/ # some output grub rescue> ls (hd0,msdos4)/boot/ # some output
We finally found the above mentioned files in the (hd0, msdos4) partition. This was the great success after searching through the internet and several wrong attempts for a long time. Now, we have to set the root to the partition which contains the kernel image using the set command so that it always points to the right location.
grub rescue> set root=(hd0,msdos4)
Next thing to be done is to run the kernel image. So for that we need to import the linux module using the insmod command and also the initramfs-linux.img file using the initrd command:
grub rescue> insmod linux grub rescue> linux /boot/vimlinuz-linux root=/dev/sda4 grub rescue> initrd /boot/initramfs-linux.img
And finally the boot command, which would boot the system(using the above files) and allow us to get into the system
grub rescue> boot
Yay! finally we were able to enter into the system. We were feeling like a cracker who has solved some crucial problem(just kidding!). But yet the problem existed ie the normal module isn’t present. In case if we would reboot the system, same error would’ve been shown and we have to repeat the same procedure. So to resolve it we need to do the same thing which we did for the 1st case. Do you remember it? Yeah, we need to re-install the grub to repair the damaged or deleted files using the system terminal.
$ sudo grub-install /dev/sda
As soon as the installation successfully completed, we reboot the system to check if it actually worked(we both crossed our fingers at that moment). And Hurrah! we have successfully solved the problem.
We both were appreciated by the people there. It was really an amazing experience. Cocoa was our mentor during the session and helped us in understanding things and showed us the path to solve the problem. I really look forward to attend more of these sessions. I think you should also. So, watch the LinuxChix space for more of these session.
References:
For now, bidding you goodbye. Meet you in the next blog. Till then…
Be curious and keep learning!
Extremely fascinating!
Would love to read more chaos posts!
Keep tinkering. Keep writing
LikeLiked by 1 person
All the commands were working in grub rescue but when i set root and after that insmod linux then it showed “linux.mod not found”. Help please…
LikeLike
Are you sure, you have set root to the correct partition?
LikeLike
Thanks a lot !
LikeLiked by 1 person
Spot on with this write-up, I actually assume this web site wants rather more consideration. I抣l most likely be once more to learn much more, thanks for that info.
LikeLiked by 1 person
It was really great to know that it helped you! Thanks a lot. 🙂
LikeLike
Thanks a lot ! =D I thought I would have to reinstall all my systems =’/
LikeLike
Sir, I found my root at hd0,mados5 but after set root and prefix but after insmod it saying not found. So any solution?
LikeLike
Which module did you try to import with insmod command? Is it linux or normal?
LikeLike
Hey, thankyou so much for article.
I have tried all ways possible for finding images but none of the partition contains those images, my partition is hd0,msdos5.
Can u help.
LikeLike
Hi,
I have a msdos1 xfs and msdos2 filesystem unknow.
So I am pretty sure that the system is on msdos1.
But : ls (hd0,mdsos1)/ result to ./ ../
Any idea ?
LikeLike
Error :invalid arch-dependent ELF magic.
When the command ‘insmod normal’ is entered
LikeLike
Sorry, I am not sure how would you resolve it. That must be some arch related module error. Have you tried searching for it though?
LikeLike
Sir, when i set root and after that insmod linux (in correct partition) then it showed “no such partition”. Help please…..sir
LikeLike
Thank you for sharing….
You help me recovered my file..
Especially, My Bachelor’s Thesis.
LikeLiked by 1 person
Sir, I stuck at another problem… normal.mod as well as linux.mod not found…… please help me to resolve this
LikeLike
You explained everything bur unfortunately nothing worked for me. Thanks for your efforts.
LikeLiked by 1 person
This is so resourceful. I can’t thank you enough for such a great help. This material rescued me from grub issues after a few trials of other resources. Thanks 😊. It’s awesomely helpful.
LikeLiked by 1 person
It was great to hear.
LikeLike
This didn’t work for me. I found the vmlinux file but insmod linux returned: error: file ‘/grub/i386-pc/linux.mod not found.
LikeLike
THANK YOU SOOOOOOO MUCH.
NEVER FOUND
Vimlizuz
Initrampfs
Img.
Or
Boot i386pc normal mod
But I had grub rec.
Boot/
And . Grub/
I was soooo frustrated..
U put me back on track.
Ubuntu 16.04.6 lts
Out.
LikeLiked by 1 person
Awesome, it worked for me as hell. Thank you so much
LikeLike
Glad to know that 😄
LikeLike
Hey I’ve problem in grub in ParrotOS.
The normal.mod is present in my grub.
The problem is:
error: file ‘/@/boot/grub/i386-pc/verifiers.mod’ not found.
LikeLike
It shows ‘grub_register_command_lockdown’. Can you help me please?
LikeLike
Brilliant, got me out of trouble following abortive install of LM.
LikeLike
Yes! It was really helpful!
Thanks a lot!
LikeLike
“We need to do this for every partition. If we get the required module in any of the partition. Let’s say we get it in the 1st partition. So, now we need to set the prefix variable to the partition which contains the required file.”
What if we don’t? Under Win I see normal.mod on Ext2 partition, but grub rescue don’t.
LikeLike