23 mars 2007
LP64 or the great return of segmented-mode
LP64 is the commonly choosen 64bits C mode, where integer are 32bits, and Pointers or Long are 64bits.
This mode is the evil itself, as this is the great return of segmented mode itself!
First of all, why LP64 is similar of the 32bits segmented-mode:
LP64 is used to have good compatibility with 32bits C-code, enabling re-compiling in 64bits mode without too much side-effects. The idea is to enable C-code to run almost identically on 64bits mode without rewriting.
As integers stay 32bits, long stays 64bits and only pointers are promoted from 32bits to 64bits, there should be no problem (other than pointers stored as integers that everytime create a warning in 32bits, and an error in 64bits LP64 mode).
So you could compile in LP64 64bits mode C-code initially written to run on 32bits mode. Almost flawlessly.
But are we in real 64bits mode or in 32bits segmented mode?
Theorically 64bits pointers allow us to manage Gigabytes or Terabytes of memory at-once without any problem! So there's should be no point?
The comes from the memory allocation, malloc(), because this often called function await for an size_t type argument. Each time you want to allocate memory, you use malloc, alloc, other another flavour of it and you end up giving a size_t typed value. The problem is, size_t is not 32bits or 64bits, not an integer or a long in C, it depends on your implementation and if you compile in 32bits or 64bits mode.
Typically in 32bits mode, size_t is an integer, so a 32bits program handle calculation of allocation size in integers, limiting them to 2GB or 4GB at once. And in LP64 64bits mode, size_t is a long, enabling theorically illimited memory allocation. BTW, as 32bits code use integers to calculate memory allocations sizes, to pass it as a size_t argument, recompiling it to 64bits mode oesn't change anything as integer is limited to 2GB or 4GB too (signed/unsigned).
So you end-up with LP64 64bits C-code, that could only allocate 2GB or 4GB at once, the same ability that has 32bits segmented code. As in 32bits segmented-code, you could not calculate pointers differences and store them in an integer.
And an example of why this is evil, with MySQL.
A main structure for MySQL is the index buffer, that caches pages of index in memory, to speed-up searches and lookups. This structure as a size given in Bytes.
If you compile MySQL in 32bits mode or use a mysql AB binary, you have the ability to use up to nearly 3GB for this structure (given you are in 3GB application-mode in Windows or Linux). In fact Typical limit on 32bits system is between 1GB and 2GB.
If you compile MySQL in 64bits mode, you have the ability to allow up to 4GB to this structure. Yes it's only twice the practical limit in 32bits mode, and far less than many 32bits OS and CPU support from years.
So if you buy a real 64bits personal computer, say an Apple MacPro, or an old Apple PowerMac, packed with 8GB or 16GB RAM, you could only allow 4GB to thye main mysql buffer, in the LP64 64bits mode!
If you use a 64bits server, packed with 64GB or 128GB RAM, you could only allocate 4GB to this main mysql buffer, ending with having instances of MySQL instead of one efficient big!
So you need tens or more instances of mysql to use your main memory on 2006 available 64bits servers, how will it evolves in '07 and thereafter?!? Hundreds of instances?
The end of the story is, when you have 64bits servers with whtaver RAM you put in it, mysql is limited to 4GB for key_buffer_size:
"The maximum allowable setting for
key_buffer_size is 4GB. The effective
maximum size might be less, depending on your available
physical RAM and per-process RAM limits imposed by your
operating system or hardware platform." (link on the mysql 5.1 documentation)
Conclusion
LP64 64bits mode, is no 64bits memory at all, and even for most known open-source software, limit RAM usage to twice 32bits mode, leaving no possibilty other than to rewrite code to enable good use of the hardware platforms. LP64 is adead-end, as was 16bits and 32bits segmented modes.
24 janvier 2006
64bits
Depuis l'arrivée des premiers microprocesseurs 64bits, toute la question de leur entrée sur les différents marchés se pose. Recherche et super-calculateurs, serveurs, ordinateurs individuels professionnels ou dans l'ordinateur à la maison, voire la console de jeu...
Le problème n'est à la limite pas de savoir si les processeurs 64bits investiront ces marchés, ils le feront.
La vraie problématique est le gain pour l'usager ou le consommateur, tant au niveau des performances que des fonctionnalités apportées.
Beaucoup de choses se disent ou s'écrivent autour du 64bits, dont AMD est le porte-drapeau aux niveau serveurs, PC d'entreprise et même grand-public. Peut-être avez-vous déja un ordinateur portable 64bits sans le savoir?
Vous trouvrez donc une série d'article sur le 64bits, les processeurs et leurs performances 32bits & 64bits (ou comment iNTEL s'est fourvoyé!), l'arrivée du 64bits, Windows et le 64bits (savez-vous qu'il existait une version 64bits de Windows, il y a déja 10ans?), les outils de développement, le potentiel, etc.