| View previous topic :: View next topic |
| Author |
Message |
xhakerek
n00b
Joined: 27 Oct 2008
Posts: 54
Location: Korolówka
|
 Posted: Mon May 15, 2017 3:57 am Post subject: Gentoo installation script with LUKS support. |
 |
|
Hello,
I'm not entirely sure if I'm puting this in a right place. Please let me know...
I have been working on something for past few months, it's still crude but I thought it might be worth sharing at this stage.
Basically it is a huge bash script, its goal is to simplify daily life and update process when using a LUKS encrypted machine. It took a few months of my life and works fine at this point. I'm still developing it but got kinda tired and thought maybe somebody will be willing to help.
Creating keys is the first step:
- USB drives are identified by serial number(serial_keys, serial_boot, serial_root)
- keys are kept on a USB drive(serial_keys) and can be added/removed(hardened=no/yes)
- keys can be backed up to a secondary drive
After keys are created(name=serial of root device), the script allows to create a bootable encrypted USB device(serial_boot). This drive is created automatically and contains GRUB with a LUKS encrypted partition.
When you boot from this device, GRUB asks for password and dectrypts the partition on which kernel resides. Kernel has a ramdisk built in, completely custom.
this initramfs attempts to open the serial_root device as LUKS and scan it for LVM label/VG/LVs. If no such content is found it can be re-created and gentoo installed or booted from RAMdisk(just stage and portage).
There is a lot of details involved. Currently for testing files are downloaded from 10.0.0.1 and the links to newest stage file are commented out.
Feel free to use and let me know what you think. As mentioned above, I'm still developing it and would really appreciate some help.
---
[1] cu_boot - https://github.com/henryk-radoslaw-rychlik/cu_boot/blob/master/cu_boot.sh |
|
| Back to top |
|
 |
Maitreya
Guru
Joined: 11 Jan 2006
Posts: 441
|
| Posted: Mon May 15, 2017 6:16 pm Post subject: |
|
|
I have so many questions.
Who is Rachel?
What are all those hashes for?
Why did I just learn something about the economics of the Deutsche Mittelstand?
What are those echos for into initramfs?
Does it build for normal PC or ARM?
You might want to split it up. |
|
| Back to top |
|
|
xhakerek
n00b
Joined: 27 Oct 2008
Posts: 54
Location: Korolówka
|
| Posted: Tue May 16, 2017 1:02 am Post subject: |
|
|
Who is Rachel?
GF
What are all those hashes for?
Hashes? Do you mean lines like this:
#CONFIG_NOUVEAU_DEBUG=y
Kernel configuration options for .config file. They basically define what is enabled in the kernel. The commented out ones are just leftovers and I will slowly split them in categories or remove.
Why did I just learn something about the economics of the Deutsche Mittelstand?
No idea how I got it there to be honest.
What are those echos for into initramfs?
Since this was intended to be a single file from the beginning, I use echo to create init script and that's what the longest echoed string is.
Init script is what gets executed after the kernel boots(either from initramfs or actual OS installation). In this case it is included in initramfs which built into the kernel image. After the script creates a bootable USB drive the boot goes like this:
GRUB -> decrypt first partition with user provided password -> load kernel image -> boot it -> kernel executes /init from initramfs -> init takes over, sets up network, checks the disks for LUKS and tries to decrypt them, checks for LVM label/VG/LV or asks if you want to download and install gentoo in ramdisk/on hdd/ssd if no OS found -> switch_root boots installed OS
Does it build for normal PC or ARM?
It started with Gentoo on used NAS I bought for $50. The goal is to support both. Bottom of the file is mostly the stuff I wrote for NAS.
You might want to split it up.
Maybe at some point. |
|
| Back to top |
|
|
Mr. T.
Guru
Joined: 26 Dec 2016
Posts: 477
|
| Posted: Sun May 21, 2017 9:42 am Post subject: |
|
|
xhakerek, the reading of your script is easy.
I wonder if the script might be reduced if it were implemented in Python? I do not know yet (I am learning Python and I continue my learning of GNU-Linux.).
I think we can remove a lot of function calls of the function named "verbose" by using the power of Python to detect errors.
Moreover, objects would also reduce some indentation errors. It would be interesting to implement this script in Python! |
|
| Back to top |
|
|
xhakerek
n00b
Joined: 27 Oct 2008
Posts: 54
Location: Korolówka
|
| Posted: Tue May 23, 2017 1:15 am Post subject: |
|
|
Hello,
I do not know Python. I'm sure it can be rewritten but honestly, I do not think it is necessary. The script itself relies on many GNU/Linux programs, porting to Python won't change it.
Take a look at set_variables function. I need higher level of verbosity for development and troubleshooting possible problems at boot. After the settings/code is polished, verbose messages can be disabled(verbose="no").
I just uploaded a new version. In general I'm getting close to the end since the script works just fine and I need some time off. I'm getting tired after countless days starring at a computer screen at both work and home. |
|
| Back to top |
|
|
Mr. T.
Guru
Joined: 26 Dec 2016
Posts: 477
|
| Posted: Tue May 23, 2017 11:01 am Post subject: |
|
|
| xhakerek wrote: | | I'm getting tired after countless days starring at a computer screen at both work and home. |
We can imagine! 
Your script has duplicated code [if possible, each instruction should be different]:
| functions: | | cecho (x2), check_for_program (x2), check_if_empty (x2), check_status (x2), clean_up (x2), configure_terminal (x2), erase_dev (x2), exit_trap (x2), format_dev (x2), get_answer (x2), get_dev_geometry (x2), get_dev_name (x2), mount_dev (x2), open_luks (x3), run_command (x2), set_variables (x2). |
| Code: | if [ "$color" == "-blue" -o "$color" == "-green" -o "$color" == "-light_blue" -o "$color" == "-red" -o "$color" == "-yellow" ]; then
eval "echo -e -n \$${color#-}\$message\$default"
...
fi |
Some parts of your code seem unusual [the function "run_command" (line 1912) seems strange?]:
| variables: | ...
local initramfs_dirs="..."
local initramfs_files="..."
...
local initramfs_dirs="$initramfs_dirs..."
local initramfs_files="$initramfs_files..."
...
local initramfs_dirs="$initramfs_dirs..."
local initramfs_files="$initramfs_files..."
... |
| CONFIG: | echo 'CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_LINEAR=n
CONFIG_MD_RAID0=n
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=n
CONFIG_MD_RAID456=n
CONFIG_MD_MULTIPATH=y
...
' >> /usr/src/linux-NAS/.config |
and the overlapping of conditions seems to add complexity (cf. function "check_lvm", line 624).
| xhakerek wrote: | Does it build for normal PC or ARM?
It started with Gentoo on used NAS I bought for $50. The goal is to support both. Bottom of the file is mostly the stuff I wrote for NAS. |
| Code: | Line 130 verbose "Creating $initramfs_temp/init"
Line 566 verbose "Creating $initramfs_temp/init" "-blue" |
You should write the code in several files because of the complexity.
| xhakerek wrote: | | The script itself relies on many GNU/Linux programs, porting to Python won't change it. |
Python has some features to interact with the operating system. I wish to implement your script in Python to know how to do it. |
|
| Back to top |
|
|
Mr. T.
Guru
Joined: 26 Dec 2016
Posts: 477
|
| Posted: Tue May 23, 2017 11:27 am Post subject: |
|
|
I gave the locations of the functions.
| Code: | build_kernel, build_NAS, get_keys, create_dirs, create_initramfs, check_lvm, install_busybox, install_gentoo, init_array, init_net, mount_filesystems, mount_root, open_devices, restore_root, run_gentoo, verbose, create_partitions, del_key, gen_key
cecho (x2), check_for_program (x2), check_if_empty (x2), check_status (x2), clean_up (x2), configure_terminal (x2), erase_dev (x2), exit_trap (x2), format_dev (x2), get_answer (x2), get_dev_geometry (x2), get_dev_name (x2), mount_dev (x2), open_luks (x3),
run_command (x2), set_variables (x2).
(line 3) build_kernel():
(line 38) build_NAS():
(line 133) get_keys():
(line 141) open_luks():
(line 180) rescue_shell():
(line 292) cecho():
(line 320) check_for_program():
(line 328) check_if_empty():
(line 347) check_status():
(line 370) clean_up():
(line 416) configure_terminal():
(line 445) create_dirs():
(line 461) create_initramfs():
(line 569) cecho():
(line 597) check_if_empty():
(line 616) check_for_program():
(line 624) check_lvm():
(line 751) check_status():
(line 774) clean_up():
(line 820) configure_terminal():
(line 849) erase_dev():
(line 893) exit_trap():
(line 906) format_dev():
(line 957) get_answer():
(line 965) get_dev_geometry():
(line 992) get_dev_name():
(line 1062) install_busybox():
(line 1077) install_gentoo():
(line 1124) init_array():
(line 1133) init_net():
(line 1181) mount_dev():
(line 1223) mount_filesystems():
(line 1240) mount_root():
(line 1268) open_devices():
(line 1289) open_luks():
(line 1330) restore_root():
(line 1376) run_command():
(line 1406) run_gentoo():
(line 1438) set_variables():
(line 1458) verbose():
(line 1513) create_partitions():
(line 1530) del_key():
(line 1558) exit_trap():
(line 1572) erase_dev():
(line 1616) format_dev():
(line 1667) gen_key():
(line 1725) get_answer():
(line 1733) get_dev_geometry():
(line 1760) get_dev_name():
(line 1829) mount_dev():
(line 1871) open_luks()
(line 1912) run_command():
(line 1943) set_variables():
|
helecho.
Last edited by Mr. T. on Thu May 25, 2017 1:11 pm; edited 1 time in total |
|
| Back to top |
|
|
xhakerek
n00b
Joined: 27 Oct 2008
Posts: 54
Location: Korolówka
|
| Posted: Thu May 25, 2017 4:02 am Post subject: |
|
|
| Quote: | | Your script has duplicated code [if possible, each instruction should be different]: |
They aren't really duplicated, it is not a mistake. Take a look at lines 131 and 214, the lines inbetween get echoed out to init file which later gets built into initramfs. I do not know of any other way to reuse the same functions. This is why some functions appear twice in the file.
131 echo '#!/bin/busybox sh
...
214 exec switch_root /mnt/root /sbin/init' > $initramfs_temp/init
| Quote: | local initramfs_dirs="..."
local initramfs_files="..." |
This part defines what files are copied to initramfs(ramdisk with init mentioned above) from the OS installation the script is running on. Since you can not copy to a non existing directory, the directory structure is defined as well. This is crude and uses variables, at some point I'll clean it up and add a function that creates the directories automatically when a copy operation fails.
| Quote: | | and the overlapping of conditions seems to add complexity (cf. function "check_lvm", line 624). |
I do agree. It took some time to get this one right. I considered splitting it up, as mentioned before, this is still work in progress.
| Quote: | | Python has some features to interact with the operating system. I wish to implement your script in Python to know how to do it. |
Go ahead, I'm not against it in any way. The code is free to use. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Thu May 25, 2017 7:30 pm Post subject: |
|
|
| Quote: | | lines inbetween get echoed out to init file which later gets built into initramfs. I do not know of any other way to reuse the same functions. |
Perhaps this would do the trick:
| Code: | cmd=sl
echo running command ">$cmd<"
$cmd
|
Also, I had some situations where I used highly dynamic code in a manner like
| Code: | | some_function_generating_commands | some_interpreter |
Handy thing* when that interpreter is an interface for some weird app rather than an actual shell, so the language it speaks is not Turing-complete and you want to run a few thousands of commands.
With any sane interpreter you will likely do better running commands directly, though there are some corner cases (lol, pun). This thing below will not work the way one could expect, though the actual behaviour is easily explained once you know what happens.
| Code: | pattern="p1 | p2 | p3"
case X in
$pattern ) code here ;;
esac |
* hated: zimbra |
|
| Back to top |
|
|
Mr. T.
Guru
Joined: 26 Dec 2016
Posts: 477
|
| Posted: Fri May 26, 2017 9:38 am Post subject: |
|
|
I think you may create the files in your initramfs in a clearer way: see /usr/src/linux/Documentation/filesystems/ramfs-rootfs-initramfs.txt
We have no need for duplicated functions: you may write the functions in a file and load this file when you need to use the functions.
In other words, you may write your script and write your functions in another file.
N.B: I will not implement the script because I didn't find the hook. I'm going to implement with other views. |
|
| Back to top |
|
|
xhakerek
n00b
Joined: 27 Oct 2008
Posts: 54
Location: Korolówka
|
| Posted: Fri Jun 02, 2017 4:18 am Post subject: |
|
|
I came up with a better solution.
The name of the script file can be checked in the script itself(@0), therefore the behavior can be adjusted based on it. This eliminates the need to use two separate files for cu_boot.sh and init(echoing the functions).
| Code: |
#! /bin/busybox sh
same functions for cu_boot.sh and init
if [ "$0" == "/init" ]; then
init workflow
elif [ "$0" == "/init" ]; then
cu_boot.sh workflow
...
cp $pwd/$0 $initramfs_directory/init
...
else
echo error
exit
fi |
[Moderator edit: added [code] tags to preserve output layout. -Hu] |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Installing Gentoo
