| View previous topic :: View next topic |
| Author |
Message |
fpemud
Guru
Joined: 15 Feb 2012
Posts: 352
|
 Posted: Thu Feb 18, 2016 1:40 am Post subject: Gentoo (Public Cloud) Build Server? |
 |
|
I'm encountering many glitches when using distcc and distcc-pump.
For example some packages (cmake) don't support distcc and some packages (grub) fail to build.
So I'd like to establish a Gentoo Build Server, which can do the whole building steps so I can install the binary files directly.
I come up with a rough idea:
1. create a virtual machine on the Gentoo Build Server, which has the same architecture as the client machine.
2. sync all system files of the client machine up to this virtual machine.
3. enable FEATURE buildpkg on the virtual machine and emerge the specified packages.
4. download all the tbz2 files create during emerge process to client machine.
5. emerge on client machine with option --usepkgonly.
When using distcc, the client machine still needs to do configure and link process.
With this Gentoo Build Server, the client machine don't need to do anything, which gives some very low end hardware a chance for using Gentoo. (For example one of my machines which has only 512MB memory)
Setting up this environment manually would be very tedious. And it won't give a good user experience (such as progress, error report).
Is it worthy to create a project to automate the steps above? (brings change to portage)
And is it worthy to create a Gentoo Public Cloud Build Server to serve all people? |
|
| Back to top |
|
 |
timeraider
n00b
Joined: 27 Jul 2015
Posts: 41
|
| Posted: Thu Feb 18, 2016 2:03 pm Post subject: |
|
|
Well, I like the idea, but I am afraid it would not work out because of all the different configurations used by different people. For example, some workstations have nvidia graphic cards while others use Intel ones and so on. How should a public build server deal with this? Should we enable all use flags?
Maybe it would be possible to sync the portage system files like proposed in step 2.). But some packages will fail during execution when configuration files are not been installed correctly (e.g., call quick-pkg without the argument --include-config, install binary packages and you may see what I mean). Uploading configuration may yield to security issues when passwords are stored inside the configuration files.
But maybe I understood you in a wrong way or you already have some ideas about this. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54872
Location: 56N 3W
|
| Posted: Thu Feb 18, 2016 6:34 pm Post subject: |
|
|
fpemud,
| Quote: | | 1. create a virtual machine on the Gentoo Build Server, which has the same architecture as the client machine. |
It works but its horribly slow as the guest CPU is emulated in software.
My brief attempt at emulating SPARC on a 3.2GHz AMD64 install got me an emulated 200MHz SPARC CPU.
I gave up and went back to my real 400MHz SPARC.
Obtaining near native speed in a VM depends on allowing the guest to execute on the real host hardware.
That's what KVM, VBox, VMWare etc all do.
The theory is sound though. You end up with a BINHOST which you can use directly if you share it over your network and tell portage about it in make.conf.
Its only useful if the packages are built with the USE flags you need, or can tolerate.
There are a few more gotchas, like the C++ ABI version too.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3507
|
| Posted: Thu Feb 18, 2016 8:22 pm Post subject: |
|
|
| Quote: | | And is it worthy to create a Gentoo Public Cloud Build Server to serve all people? |
This idea keeps coming back and back again. Typical answer is "and how would we know what source you build?" and "I want other use flags than you" or "i have different CPU" |
|
| Back to top |
|
|
fpemud
Guru
Joined: 15 Feb 2012
Posts: 352
|
| Posted: Fri Feb 19, 2016 8:34 am Post subject: |
|
|
| Quote: | | Well, I like the idea, but I am afraid it would not work out because of all the different configurations used by different people. |
In Step1, there would be one VM per client. So we can have a system exactly the same as the client. (not exactly, please see the next post)
This method can solve the "what source", "other use flags", "different CPU", "C++ ABI version", "other CFLAG", "my own patch", etc.
For Gentoo Build Server, a regular PC is good enough to host 3-5 VMs.
For Gentoo Public Cloud Build Server, it needs to be a powerful one, or even more complex, a cluster system, to hold tens of thousands of VMs.
| Quote: | | It works but its horribly slow as the guest CPU is emulated in software. |
It would be more complex on the server side if the client and the server have different arch, but it is transparent to the client.
Aside from creating a VM (name it worker-vm) that has the same arch and system files as the client, another VM (distcc-vm) is needed for distcc-cross-compiling.
worker-vm is slow, but it only needs to do configuring and linking.
All the compiling burden is on the shoulder of distcc-vm, which can run at near native speed.
I think this is just what you do if you run Gentoo on a low end SPARC, MIPS or ARM system.
| Quote: | You end up with a BINHOST which you can use directly if you share it over your network and tell portage about it in make.conf.
Its only useful if the packages are built with the USE flags you need, or can tolerate. |
I don't want the BINHOST only provide some general binary package.
I'd like to reach the following user experience:
This BINHOST just has the tbz2 files you need, and astonishingly these tbz2 files have no difference with your own build result.
My question is: Is it technically possible?
| Quote: | | But some packages will fail during execution when configuration files are not been installed correctly |
Sorry, I don't get it.
I think with my steps, the configuration files should be installed correctly.
| Quote: | | Uploading configuration may yield to security issues when passwords are stored inside the configuration files. |
Fortunately it is no problem for a Gentoo Build Server, which only run in local network.
It should be concerned before we build a Gentoo Public Cloud Build Server. I believe solution exists.
| Quote: | | This idea keeps coming back and back again. |
Is there any link? I'd like to study it.
Last edited by fpemud on Fri Feb 19, 2016 9:26 am; edited 11 times in total |
|
| Back to top |
|
|
fpemud
Guru
Joined: 15 Feb 2012
Posts: 352
|
| Posted: Fri Feb 19, 2016 8:57 am Post subject: |
|
|
| Quote: | | In Step1, there would be one VM per client. So we can have a system exactly the same as the client. |
It should be more complex.
The worker-vm's boot-loader, kernel, init system and system services is definitely different from the client.
Maybe we should sync client's system files into a directory of worker-vm, and emerge in a chroot environment.
I think this won't affect the build result. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9626
Location: beyond the rim
|
| Posted: Fri Feb 19, 2016 11:39 am Post subject: |
|
|
| fpemud wrote: | This BINHOST just has the tbz2 files you need, and astonishingly these tbz2 files have no difference with your own build result.
My question is: Is it technically possible? |
If you mean absolutely no difference at the binary level: No.
If you mean no functional difference: Possible, but tricky to ensure.
Thing is: cost/effort ratio to create and/or operate such a system is not viable. Especially if you're considering public where security and privacy issues come into play. And for local networks the easier and more reliable option is to simply mount the client filesystem on a buildhost and emerge normally in a chroot.
The general idea has been discussed years ago: http://www.realgeek.com/forums/gentoo-dev-remote-package-building-service-354222.html |
|
| Back to top |
|
|
fpemud
Guru
Joined: 15 Feb 2012
Posts: 352
|
| Posted: Sat Feb 20, 2016 12:09 pm Post subject: |
|
|
| Quote: | | If you mean no functional difference: Possible, but tricky to ensure. |
I think this level is enough.
| Quote: | | And for local networks the easier and more reliable option is to simply mount the client filesystem on a buildhost and emerge normally in a chroot. |
I tried this method today using net-fs/curlftpfs, the speed is very very slow.
NFS would bring better performance, but I think it is too heavy for this job. Besides I have disabled NFS in my kernel config long time ago.
I think the "sync to server" and then "build in chroot" is simple enough and can fulfill my current need.
I'd like to create a project using this solution. I'll report any progress here. |
|
| Back to top |
|
|
timeraider
n00b
Joined: 27 Jul 2015
Posts: 41
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
