# Using the new compiler in production YAPC EU, Granada Sep 2, 2015, 14:30 20 min Reini Urban, cPanel --- # cPanel Running compiled perl binaries on 70% of the webhosters world-wide, with Centos 5-7. --- # cPanel Running compiled perl binaries on 70% of the webhosters world-wide, with Centos 5-7. Centos 6 mostly. Providing the SW stack for them, on top of Centos. --- # History System perl on centos usually 5.8.8 or 5.10.1 cPanel perl 5.14.4 cPanel compiled perl: 5.6.2 => 5.14.4 (since 2014) --- # History Perl 5.6.2 2001 - 2014 compiled with hash randomization Stable B::C compiler, in core until 5.8.9, broken with 5.10. Custom patches, optimized linker, see my previous YAPC::EU 2010 in Pisa talk. --- # Criteria for cPanel * **memory** no memory hog on 500MB customer VMs * no utf8 just pass translations through, use bytes. * compiled 10-20% less memory, better startup, license --- # Memory  --- # Memory  --- # Memory  --- # Characteristics A typical cPanel binary is 50MB has ~200 module dependencies has almost zero startup-time perl5 bound run-time and again almost zero destruction-time --- # Characteristics * 10-30% less memory * 10-20% better startup time * 100% faster destruction time --- # Characteristics Most modules are already compiled in Shared libs are loaded at startup Some bigger modules (XML) not constantly used are run-time loaded from source *(not via `.pmc` or `.so` yet)* No compilation to extra shared modules yet --- # Characteristics ##Before Resident Size (perl 5.6)_____9756 Resident Size (perlcc 5.6)___9072 (7%) ##After Resident Size (perl 5.14)____18704 Resident Size (perlcc 5.14)__12091 (35%) --- # Why 35% better? Aggressive string optimizations with -O3, from dynamic to static. --- # Why 35% better? Aggressive string optimizations with -O3, from dynamic to static. The compiler is also now 2-3x faster than the old compiler. The build process was reduced from 2 hrs to 40 minutes. --- # Why 35% better? Aggressive string optimizations with -O3, from dynamic to static. But the problem is more why 90% worse with 5.14 --- # Development First compiler version 1.42 (Feb 2012) with 5.14 could compile and run all cPanel code. 3 months work after moving to Texas. Worked on several optimizations and 5.18 support until 2014, when cPanel was ready to switch over. Still 5.14 (and 5.18 in test). 1.44 (Feb 2014) added the full core testsuite, and passes it. Before we had ~20 fails. Problems: - global destruction (please use lexicals) - attribute handlers - compile-time perlio layers --- # Development Until 1.52, the final version Sep 2014, we fixed: - global destruction (rewrite) - Dynaloader/XSLoader handling - tons of corner cases detected in the testsuite (added mro, utf8, re tests), and with modules - ~5 CPAN module fixes esp. DBI, Encode, Net::DNS, IO::Socket::SSL, DBD::mysql for compile-time stored pointers, being restored at run-time, boom! --- # Beware Still the old golden rule for the compiler: **BEGIN for compile-time, INIT for run-time initializations.** Esp. do not compile pointers into your shared library, initialize them at **INIT**. --- # Testing The core testsuite runs now all tests in *12:40min* fully parallel. With ccache *~5min*. Before it needed *16min* for 75% of it. --- # Testing The core testsuite runs now all tests in *12:40min* fully parallel. With ccache *~5min*. Before it needed *16min* for 75% of it. * our uncompiled perl 5.22 variant needs *8m40m*, * a p5p perl 5.23 needs *11-13min*, * 5.14 *15-19min* --- # Broken Moose, Coro, FCGI, v5.22 We'll rather fix perl 5.22 and B::C first, and make the Moose bloat history. I maintain in p2/potion a perl6 like MOP in ~40 lines, which is 200x faster. Fixing Moose is not priority, fixing perl5 is. Coro next, when we'll start using it. --- # Development Via perl11 in nearby Austin we formulated goals for perl6 feature integration. * types (YAPC::EU 2012 Riga talk) * B::CC optimizations (YAPC::EU 2013 Kiev talk, ~6x faster) * p2/potion development (YAPC 2014 talks, ~5-200x faster) * rperl (restrict the syntax, map to static C++ types, ~7-350x faster) * plan on shared modules, less memory, better arrays and hashes and types (=> no magic, static optims and dispatch) and much more. --- # Next goal We skipped all versions from 5.16 to 5.20, since they got worse in almost all aspects: memory, security, compiler API. But with the monomorphic inline cache patch by Oleg Pronin early 2015 and the :const attribute in core (for anonsubs, but we took the syntax), 5.22 became the next realistic target to aim for. --- # Next goal Even with several major blocks (p5p, unicode bloat, COW, SVf\_PROTECT, @\_ in sigs), I started working on a fixed and improved perl version in Feb 2015. <!-- Starting with my early 2012 type system work which had no chance to get accepted or even discussed with p5p.--> :const made it possible. --- # Next goal Compiling >5.14 with perl5 proper is not worthwhile. 5.22 maybe without COW. But targetting our improved version based on 5.22 and coretypes is. We can control memory bloat, overcome the broken p5p development status <!-- which is going on since 2002, add the needed features which were not added since 2002, --> and can use the improved compiler (op.c) and perl run-time in the B::C compiler to produce better code. Back to the 5.6.2 level hopefully. Memory-wise and initialization already there, but we still have some major improvements in not yet merged branches. You can expect a **perl11 (5+6=11)** soon. But we prototype and use it internally for a while first. --- # Questions