Compiling help

[MRD]

Interesting, I think what he's saying is to keep two package directories on the file server. All machines build packages and put them in the package dir, but their idea of what the package dir is is different, so tbirds will put them in one place, and xp's in another.

Here's the question though. When you build a binary package, is that stored somewhere so that it now knows that there is a binary package built for that ebuild? If so, say you had built it on the XP system, but not the tbird system, would the tbird system go looking in the tbird package dir, and not find it?

How does it determine if there is a package built or not? Does it just look in the directory, or in a file in the package directory?

The other problem I run into sometimes is that I need to make my use flags more consistent between systems. I add them rather haphazardly. I could create a dynamic link for make.conf to a single make.conf on the file server, although I'd need at least 2, one for each -march. I guess it wouldn't matter if use flags differed at all between the two arches anyways, since they can't share packages.

 

[splat]

hmm. i'm wondering if you could simply set up the $SYSROOT directories to be different, even if they are the same CHOST, so you can keep the CFLAGS seperate.

so, you'd be able to use the same crossdev environment, but have /usr/i686-athlonxp and /usr/i686-tbird, then you'd have /usr/i686-athlondxp/etc/make.conf with one set of CCFLAGS and /usr/i686-tbird/etc/make.conf with another set of CCFLAGS...and each /usr/* would have their own compiled binary dir. then you share those binary directories over NFS, and mount the appropriate one on each client.

 

[MRD]

I don't understand enough about the guts of portage to make this work, but I'm pretty sure if I did, it would be easy.

 

[splat]

well from what i'm getting from http://www.gentoo.org/proj/en/base/embedded/cross-development.xml, crossdev just sets up the toolchain to match the CHOST of the target system, if different from the host system. Like i run x86_64-pc-linux-gnu on my desktop, but if i want to compile things for my 32bit athlon xp, i need to setup crossdev to install a toolchain for the 32bit i686 target environment. If every system in you farm runs i686 then you don't need to setup crossdev, but I think you can still set up different portage directories for the different CFLAGS that you would run.

so say you have a k8, an athlon xp, and thunderbird. k8 would be the host that compiles everything, so you'd make /usr/i686-athlonxp and /usr/i686-thunderbird, create /usr/i686-athlonxp/etc/make.conf with all your athlonxp specific stuff (including binary dir), and create /usr/i686-thunderbird with all the thunderbird specific stuff (including binary dir). then copy /etc/make.globals to /usr/i686-*/etc/make.globals, then link /usr/i686-*/etc/make.profile to the correct /usr/portage/profiles/ file. then you'd have to create 2 seperate emerge wrappers, one for the athlonxp and one for the thunderbird, in a style similar to what's described in "xmerge: A simple cross emerge wrapper" on that crossdev page, so you might want a athlonmerge and a thunderbirdmerge file, each with their PORTAGE_CONFIGROOT pointing to the appropriate /usr/i686-* dir.

if you don't want to use the wrapper scripts, you can just use something similar to"
CBUILD="i686-pc-linux-gnu" ROOT="$SYSROOT" PORTAGE_CONFIGROOT="$SYSROOT" emerge -av ntp
to emerge things.

and there is:

Preparing cross-compiled packages for deployment with qpkg

Packages cross emerged to the sysroot can easily be packaged for deployment using qpkg.

Code Listing 4.3: Creating a binary package

# ROOT="$SYSROOT" qpkg -P "${SYSROOT}/var/tmp/binpkgs" ntp


The package is tarred, bzipped, and saved to $SYSROOT/var/tmp/binpkgs with a .tbz2 extension.

Note: The -P option was introduced into qpkg in portage-utils-0.1.17.

at any rate, trying to have two different CFLAGS for the same CHOST looks like it's a little tricky...which is why I think most binary distributions contend that setting custom CFLAGS doesn't really increase performance much.

 

[MRD]

It actually makes quite a difference sometimes. -march=athlon-xp vs -march=athlon-tbird is the difference between using SSE and not using SSE, and that can be huge.

 

[splat]

what if everything was compiled to have SSE, how would that effect performance on non-sse chips? how would it effect other things, like package size? i'd assume that binary based distros compile their packages with all options on, to cover the features availably by all processors. So if you want to distribute binaries over a large, mixed farm, you might have to take on the same mentality if you want to compile only once.

 

[MRD]

If you compile with SSE optimizations and try to run them on a tbird, they core dump. The instructions aren't there in the cpu, and it tries to use them. You might as well try to run a PPC binary on an x86, it just doesn't work because you can't call instructions that aren'tthere.

 

[splat]

so how do binary distros handle that? and windows for that matter?

 

[MRD]

There are 2 ways. One is to simply use the lowest level instruction set, the one everything has. For example, most linux distros are built for a 386, since all later processors are supersets of the 386.

Some packages, like mplayer, detect hardware on the fly, and fork into different pieces of code depending. Do an equery uses mplayer and you'll see a cpudetection useflag. This takes a bit of time when the program starts, but it detects the cpu, and decides which optimizations can be used. It's best to compile it for your particular chip, since you don't waste time detecting and you don't build in code that will never be used, but the cpu detection wll work.

Most programs don't have this though, so you are not fully utilizing your processor.

If you notice, some software requires 386 or higher, 496 or higher, pentium or higher, etc. Generally that means they built the binaries to use those instruction sets. Windows 2000 cannot be installed on a 486 (probably) because it says pentium required, which likely means they built the code using the i586 instruction set, and it will crash a 486.

Arch Linux is known for the fact that it's i686 optimized. That means that it won't run on anything less than a P2 or an Athlon classic. k6-2's, pentiums, 486's, etc will not run Arch. (PPro will though, it's i686... it was the first.)

 

[telexen]

You can specify the PKGDIR and have two different ones. I already use "/usr/portage/package/$HOSTTYPE" for mine, as I run amd64 on my desktop and x86 on my server. It works just fine.

However, I just tested this out and it works to build packages in the specified dir but GCC doesn't like the way -march was set. I'm looking for some other kind of system variable that could be used right now.

 

[MRD]

If you figure out a way to keep to binary package directories for 2 different sets of cflags, let me know.

 

[splat]

There are 2 ways. One is to simply use the lowest level instruction set, the one everything has. For example, most linux distros are built for a 386, since all later processors are supersets of the 386.

Some packages, like mplayer, detect hardware on the fly, and fork into different pieces of code depending. Do an equery uses mplayer and you'll see a cpudetection useflag. This takes a bit of time when the program starts, but it detects the cpu, and decides which optimizations can be used. It's best to compile it for your particular chip, since you don't waste time detecting and you don't build in code that will never be used, but the cpu detection wll work.

Most programs don't have this though, so you are not fully utilizing your processor.

If you notice, some software requires 386 or higher, 496 or higher, pentium or higher, etc. Generally that means they built the binaries to use those instruction sets. Windows 2000 cannot be installed on a 486 (probably) because it says pentium required, which likely means they built the code using the i586 instruction set, and it will crash a 486.

Arch Linux is known for the fact that it's i686 optimized. That means that it won't run on anything less than a P2 or an Athlon classic. k6-2's, pentiums, 486's, etc will not run Arch. (PPro will though, it's i686... it was the first.)

well I understand the difference between i386 and i686, but I thought that athlon xp's and thunderbirds were both i686. If the axp's and tbirds are both i686 (and core 2 duo's for that matter), then why would distro's like Debian compile packages for a Thunderbird that couldn't use all the new features of a C2D? And why would they claim that specifying custom cflags doesn't improve performance? I'm not attacking you, but I'm attacking all the people who say that running Gentoo and specifying custom cflags and custom compiling every package doesn't offer performance gains. Because, from what you are saying, it would make no sense to use a binary based distro, or even windows, if the binary packages couldn't detect the cpu at run time and implement special instructions like SSE on it's own.

If you figure out a way to keep to binary package directories for 2 different sets of cflags, let me know.

did my suggestion in post #44 not work?
Or could you make a dpkg wrapper script what would pass CFLAGS="" at the command line and store the binaries in a special dir, so you could have a 'axpdpkg' and a 'tbirddpkg'?

 

[MRD]

The instruction set of a tbird is a superset of the i686 set. The instruction set of the Athlon XP is a superset of that of the Tbird. There are instructions in it that are not present in the Pentium Pro (the first i686). Examples are 3dnow, 3dnow extended, and (I think) mmx extended.

i686 is a set of instructions that virtually all processors produced since the first athlon by AMD and the first PPro by Intel have included, so if you compile for that everything works.

Most of the instructions added over the years regard floating point optimizations and graphics/multimedia stuff (mmx, 3dnow, and sse). These have little to no effect on many applications.

Also, each step up gives a smaller benefit. So compiling for 486 might be a lot faster than 386, but the gain from going from 486 to 586 is noticeable but smaller, and the gain going from 586 to 686 is even smaller. The most critical instructions were implemented first. The later ones matter less. That's why some people argue that the speedup isn't that important from compiling.

The speedup, however, can be huge in some applications. Folding is an example. Foldingathome includes the ability to autodetect the cpu and use different branches of code. A cpu using i686 alone is much slower than one using 3dnow, which is much slower than one using SSE, which is much slower than one using SSE2. If you look in your folding logs, it will tell you things like "SSE optimizations on" or some such message. My Athlon XP 1500+ (1350 mhz, 256kb cache) DESTROYS my Athlon Tbird 1400 mhz 256kb cache) in folding because of the SSE in Athlon XP's.

It is true that everyone would gain performance by using a version custom compiled to their processor, but it would break backwards compatibility, so most binary software distributors go with the backwards compatibility. Some, like Arch, have decided to cut off support for anything pre i686, and that makes Arch faster than distros that continue to support cpus all the way back to i386. Also many distros include kernels compiled for i386, i486, i586, and i686. You can install them in debian with apt. Red Hat packages also include the arch they are compiled for in the name usually.

One of the main reasons i686 is used is that it's the last architecture that EVERYTHING supports. After that, AMD and Intel branched in different directions, with AMD focusing on 3dnow and Intel on SSE. It made it harder, even with the passage of time, to find an instruction set they all supported.

Finally, C2D's have a whole set of instructions that are not present in i686 computers, as well as the i686 instructions for backwards compatibility. The instruction set is usually called AMD64, although Intel uses a different name for it.

It would be interesting to pick some applications, compile them for 386, 486, 586, 686, athlon-tbird, and athlon-xp and benchmark them on some tasks to see how much difference it makes.

More and more manufacturers are discontinuing support for older cpus as 1) there is no chance you're going to be able to run World of Warcraft on a 386SX anyways and 2) not many people have them anymore. On the WoW box it says "Required Pentium III 800 mhz or higher or AMD Athlon". Most likely that means, "We compiled this package for i686 w/ MMX" It probably also has the ability to detect and use SSE on those processors that support it, as it would matter for that package. Most likely if you tried to run this on a P1, it would simply crash horribly, because the Pentium doesn't have the right instructions. It would probably run on a P2 though... albeit very badly. The only instructions (I think) in a P3 not in a P2 are SSE, and AMD Athlon classics do not have SSE, and they are supported. (It's possible they also added extended mmx then, but I don't think so.)

did my suggestion in post #44 not work?
Or could you make a dpkg wrapper script what would pass CFLAGS="" at the command line and store the binaries in a special dir, so you could have a 'axpdpkg' and a 'tbirddpkg'?

TBH, haven't goofed around with it yet. I'm a bit afraid to until I undertstand how it keeps track of which packages it has prebuilt in binary format.

 

[splat]

i understand how running a i386 distro would different from running i686, but what I am still not fully understanding is what the CFLAG settings actually do. I thought the CFLAG settings were just options passed to the compiler to tune how things were actually compiled.

if I compile software on an i686 arch, with -march=nocona in the CFLAGs, will it produce code that cannot be run on a pentium 2 running an i686 distro? I thought all the specific sse, 3dnow, mmx, etc. options were set in the USE flags.

 

[MRD]

I am not familiar with -march=nocona. I've never used that -march. I can tell you that if I swap in a tbird into my Athlon XP boxes, some programs no longer work and core dump. Others work fine. It just depends whether they use SSE, which is the only difference in the instruction sets. Note that there is only one -march for palominos, tbred-a's, tbred-b's, thorntons, and bartons. They have differences in cache size and fsb speeds and such, but they all have the same instruction sets, so they use the same -march.

If you set the compiler to -march=athlon-xp, it can use SSE in the compilations, but it may not do so if it is not suitable. For example, if you compile a word processor, it probably won't use SSE.

The useflags override those settings, but not all apps use them. Some simply rely on the basic cflags to determine build parameters.

 

[splat]

alright I think I understand it now. so the -march does actually change the compiled code. I was thinking it was just a flag for gcc to help gcc itself run faster. http://gentoo-wiki.com/Safe_Cflags has a nice intro description to explain some of the choices. By looking down the list, i'd assume that binary distros compile their packages with no -march set.

-march=nocona is for the Core architecture. that safe cflags wiki page only uses it in for the 64bit environment, but I'd read other places that -march=nocona can be used safely in both 32bit and 64bit environments...similar to how -march=k8 can be used in both.

while i was browsing thru the wiki i found this: http://gentoo-wiki.com/TIP_Providing_binary_packages

 

[MRD]

Yes, -march changes the compiled code. It will make it so the code cannot run on some architectures. If you want GCC to run faster, build it (with GCC) using the fastest -march option that your computer supports. Its speed will be determined by the compiler options used to build the compiler itself. Also, using more optimizations will generally make compilers slower. -O0 is much faster than -O3, but the code generated by -O0 is much slower than that generated by -O3.

If I could find a way to do it, I'd actually set up the file server to download and compile every single gentoo package for both arches. That way whenever I wanted to emerge something, it would be practically instant.

That would keep the file server busy compiling all the time, but who cares.

 

[splat]

If I could find a way to do it, I'd actually set up the file server to download and compile every single gentoo package for both arches. That way whenever I wanted to emerge something, it would be practically instant.

That would keep the file server busy compiling all the time, but who cares.

on your host system, try copying all your portage config stuff to a dir like /usr/i686-athlonxp, make your athlon xp specific changes to /usr/i686-athlonxp/etc/make.conf, mkdir /usr/i686-athlonxp/binpkgs, set /usr/i686-athlonxp/binpkgs as the PKGDIR in /usr/i686-athlonxp/etc/make.conf, then run 'PORTAGE_CONFIGROOT="/usr/i686-athlonxp" emerge -av --buildpkgonly somepackage' or make a emerge wrapper so you don't have to type all that stuff each time you want to build a package

on the athlon-xp client set:
PORTAGE_BINHOST="http://ip.address.of.server/path/to/BINHOST"
PKGDIR="/path/to/alternative/package/directory"

then on the client you would run 'emerge -va --getbinpkg --usepkg somepkg'