Talk:PHP 5 install script/dev

Dev Notes
In regards to backing up the php.ini file, I've not really found a good, solid solution other than how I've set it up currently.

Right now it just checks to see if the install directory, which is given by the script and does *not* check if the user had previously used another directory, has an 'etc' directory and backs up any domains listed within. Personally I find this to be a bit ugly, because it doesn't allow the user to modify the install directory if they likewise wish to use the backup feature. Not that most users would ever bother with that though I suppose... Mousee 09:40, 30 May 2007 (PDT)

Currently the default wget location is set to a gentoo ftp mirror that's in California. The mirror is fast, contains all of the dist files, and I believe is best suited for this type of script in consideration of the inherent slowness of some of the other mirrors. That said, the secondary mirrors are the originals and should work fine in case the primary mirror goes down ever. Again, if anyone has a better solution to this, please add it or discuss it here. Mousee 09:40, 30 May 2007 (PDT)


 * Use of the Gentoo FTP Mirror should be OK but they get very touchy about people doing that (I should know, I was a Gentoo Developer/Maintainer for 4 years and that caused major diplomatic problems when other distribution knockoffs of Gentoo used their mirrors). The version of the script I'm working on uses the straight GNU and SourceForge round-robin mirrors instead. -- Exsecror 05:46, 9 October 2007 (PDT)

Please also note that the addition of OSSP mm (Shared Memory Allocation) is completely experimental, as I'm not sure how detrimental it'll be to the given user memory allotment. From past experience, the memory usage is nearly non-existent, and it's use on DreamHost should work to improve overall site performance (load times specifically) as there is less dependence on the NFS file system, which is generally a bottleneck from time to time.
 * It should be noted that use of OSSP mm is not a good idea right now considering how broken it is in 5.2.3 (then again no one should be using <= 5.2.3 given the serious security holes >= 5.2.0 has) with privilege escalation exceptions. Now generally this isn't a problem since DreamHost uses GRSecurity but because there's no way of knowing of their setup, use of the module is discouraged. -- Exsecror 05:46, 9 October 2007 (PDT)

Works Well
I needed an idiot-proof solution. I know how to do the compilation, but setting the right parameters for each requirement, let alone PHP's compile process, I usually ended up a few hours taken from my life and no successful compilation in the end. This works, as far as I can tell! Stripping debug info from the cgi-bin files as I speak!...except phpize doesn't see autoconf, even though its installed and I have specifiied its location.

General
Alright Mousee as you already know from our previous discussion I'm working on another version of the script. However it's been renamed to php5-toolchain.sh, since it is in essence, a toolchain. The differences in this script are major in itself, for one the script installs everything in /home/${username}/local as per LHFS structure and will refuse to install if that location already exists. Secondly when the script is initalized it will ask the user a series of questions, namely where they want their distfiles and sources to go (it sets defaults if not specified) as well as their DH docroot path. It also asks the user if they want the Suhosin extension applied and if they want to enable PEAR/PECL support (e.g. so they don't have to use the scripts available on the wiki to install APC or PEAR locally) and will add the path /home/${username}/local/usr/bin to their ${PATH} appropriately so they can call PECL/PEAR directly. I'll also have it patch the PEAR environment file as well to avoid using /tmp.
 * What are you coding the script in? Just normal bash/shell script right? I was thinking of doing a 'plug-in' system for add-ons like Suhosin and the like, where each add-on would have their own installer file which would contain some additional code (ie. a "Plugin Name") that would be external from the package in consideration of individual plug-in updates, but I'm not entirely sure such a system would work well in bash script. It would make it easier on package maintainers to update individual add-on code, versus searching through the PHP5 installer for said code AND updating the standalone version of the add-on, and it would also make it easier to add new, future add-ons. The basic concept would be to write the "PLUGINNAME", "PLUGINVERS", and "PLUGINDESC" variables in each add-on's code which would be grep'ed into the main program (during the program's execution) and display its Name, Version, and a brief description of itself, and then call the main installer code called if it were selected. It's something to consider anyways. - Mousee 06:54, 9 October 2007 (PDT)
 * The script is a simple BASH script that handles everything. I don't think a plugin system would be necessary though since a large portion of extensions are available through PECL and if they answer yes to that question, automake and autoconf are built and the pecl binary is added to their path, same with pear. -- Exsecror 07:01, 9 October 2007 (PDT)

Security
All the binaries required in the script have been updated to their latest stable (e.g. PHP 5.2.4) and so on since under the PHP Security Consortium requirements, no one should be using less than PHP 5.2.3 due to the massive security holes in the filter extension and a few other extensions. So far only one of the libraries, libmhash, had to be patched to fix a massive security hole in it's cryptology logic but I have had yet to work on the script this week due to work.
 * Totally agreed here. Security has always been my primary concern. - Mousee 06:54, 9 October 2007 (PDT)

Overall
Some of the overall changes besides how the behavior of the script is and how it looks and feel is the way the programs are built. Dependency linking is a bit different now than it is with the current script such as libxml2 and libxslt utilizing zlib and libgcrypt for increased security and speed as well as the fact that debugging in almost all libraries and python support for some (due to the lack of it by DreamHost) has been disabled. Now because debugging has been stabled all the shared libraries and binaries are also being stripped to conserve space and reduce memory loads which will make DH happy and allow it to run somewhat quicker.

Another thing is the script will make various changes to the php.ini prior to installing and set up basic security for the user such as forcing the upload path, sessions path, wsdl cache path and so forth to directories outside of /tmp which is a good thing because you should never use /tmp for that. It will also set the memory limit to DH's setting of 90M and if they use Suhosin it will enable the forced memory limit to 90 because no well written script will ever use that much memory. It will also advise the user if they have chosen to enable Suhosin not to enable the emodifier protection option since some broken applications use the /E modifier (WordPress) however I'm in talks with most of them as they should be using preg_replace_callback, not /E. It will then also create the bootstrapping environment for FastCGI and let them know how to modify their .htaccess accordingly. -- Exsecror 05:46, 9 October 2007 (PDT)
 * Using libgcrypt is an excellent idea and I do certainly agree with stripping the (currently useless) debug info from each program. I was honestly under the impression that such info was NOT included in most of them unless explicitly compiled in as an option, but I never actually looked into that either. Your ideas with php.ini are exactly what I was doing as well and would make troubleshooting custom installs much easier. My only problem with your installer idea thus far I believe, is that you haven't, at least stated, any plans to allow for users to upgrade their existing install. Will you be providing that as an option or were you planning to go the complete re-install route? I was considering a method of keeping the user's install options, which would basically just be copied to a separate file (ie. php5_install.opt or whatever) and the user given the option to choose the settings therein or do a fresh install (which would force them to select all new options and such). It would make re-installation and/or updating PHP5 MUCH easier in my opinion. How to go about copying such options into a separate file would be a bit annoying, but certainly could be done via bash script. Everything else with your concept is either the same or better than my original design, save for the fact that mine has a menu subsystem. =p -  Mousee 06:54, 9 October 2007 (PDT)
 * There will be an update procedure for the future yes, once I get all the bugs worked out of the compile/installation process there will be manifests with sha512sums generated for each program so that upgrades are possible (ala Portage style, build then replace) but I do things one step at a time so as not to screw myself. -- Exsecror 07:03, 9 October 2007 (PDT)

Non-Domain specific installation
I have multiple subdomains and wanted to make a global setup. I was willing to copy in a specific htaccess and wrapper fcgi file as appropriate into the web space. I have modified your script to remove the reliance on the domain variable and commented out the sections that would of otherwise created the cgi-bin folder and copied the php binary there. The wrapper fcgi file can call a binary outside of the web domain so it isn't necessarily a requirement to copy the file into the web domain space. The way your script is setup though does make it useful if the user wants to have domain specific configurations however it might almost be easier to just call the php binary with a parameter to point it to a different php.ini file for that area.

I will keep track of this topic and if there are major changes to the script I will update the binary appropriately on my end. Currently compiling all dependencies for 5.2.5 (hope that 5.2.4 dependency versions will work for the new version of php). Shinji 22:55, 26 November 2007 (PST)


 * That's actually an excellent idea and one I'd thought of somehow attempting to include for some time in the script, as not all users are installing PHP5 for just a single domain (or two or three). There may be a way to create two different routines with an additional variable that will allow us to do so... I'll have to look into it more later - or you may consider adding it yourself ;) - Mousee 03:45, 27 November 2007 (PST)

Some improvements to the php5-install-prep.sh
For some reason I didn't see this PHP 4 install script/dev area and only saw the page at http://wiki.dreamhost.com/PHP_5_install_script

I see now that I spent time improving outdated code? This dev version is much farther along than the main version. I really like all the additional error-checking and fluidness of this dev version, but I think it is needs to be refactored. Since the code I came up with didn't have to mess with any of the configuration or have much to do with the main installation process I was able to make the code a lot slimmer. By using functions I've made it easier to update as its modular. I definately DO like how these 2 aspects of the install process are separate, the download and unpacking and the configuration as the other.

Anyway, hope this helps, and ya this script needs a lot of work, i'll either go dig up some of my shell scripts from years ago of designing init shell scripts for controlling boot processes on linux and throw a few more hours into this, but then again, not many people will ever want a custom php. I am really excited about what we are going to be able to accomplish with such an intelligent webhosting provider.


 * 1) !/bin/sh


 * 1) Version 0.6, 2007-12-16
 * 2) - Updated 2007-12-16 by AskApache (www.askapache.com)
 * 3)   - Implemented functions to fetch the URI and decompress it
 * 4)   - Added a couple more error-checks
 * 5)   - Replaced wget with cURL
 * 6)   - Added more to help keep it from getting killed
 * 7)   - Updated to php-5.2.3, curl-7.17.1, freetype-2.3.5
 * 8) - Updated 2007-01-15 by Charles Wiltgen (charles@wiltgen.net)
 * 9)   - Make "nicer" to help keep it from getting killed by DreamHost
 * 10)   - Make less verbose to keep signal-to-noise level high
 * 11) - Updated 2006-12-25 by Carl McDade (hiveminds.co.uk)
 * 12)   - Allow memory limit and freetype
 * 1)   - Allow memory limit and freetype

set -e
 * 1) Abort on any errors

export DOMAIN="your-root-domain.com"
 * 1) The domain in which to install the PHP CGI script.

SRCDIR=${HOME}/source
 * 1) Where do you want all this stuff built? I'd recommend picking a local filesystem.
 * 2) ***Don't pick a directory that already exists!***

INSTALLDIR=${HOME}/php5
 * 1) And where should it be installed?

DISTDIR=${HOME}/dist
 * 1) Set DISTDIR to somewhere persistent, if you plan to muck around with this
 * 2) script and run it several times!

PHP5="php-5.2.3" LIBICONV="libiconv-1.11" LIBMCRYPT="libmcrypt-2.5.7" LIBXML2="libxml2-2.6.27" LIBXSLT="libxslt-1.1.18" MHASH="mhash-0.9.7.1" ZLIB="zlib-1.2.3" CURL="curl-7.17.1" LIBIDN="libidn-0.6.8" CCLIENT="imap-2004g" CCLIENT_DIR="imap-2004g" FREETYPE="freetype-2.3.5"
 * 1) Update version information here.

export PATH=${INSTALLDIR}/bin:$PATH
 * 1) Push the install dir's bin directory into the path

function aa_unpack { # compressed, tar and gzip files to DISTDIR if [ -f $DISTDIR/$1* ] ; then echo Extracting "$1"; zcat ${DISTDIR}/$1* | tar -xvf - &>/dev/null; echo Done.;	echo; wait fi }

function aa_grab { #saves file to SRCDIR echo `basename $1` curl -L --retry 20 --max-time 1800 --retry-delay 30 -# -f --max-redirs 4 --remote-name "$1" }

echo echo -- echo --  Run this script before php5-install.sh     -- echo -- echo echo - Downloads and unpacks all prerequisite packages echo - **SRCDIR and DISTDIR will be deleted** echo read -p "        (Press any key to continue)" temp; echo;echo

if [ -d "$SRCDIR" ] || [ -d "$DISTDIR" ];then echo echo --- Cleaning up any previous attempts --- rm -rf $SRCDIR $DISTDIR &>/dev/null echo Done. echo wait fi
 * 1) cleanup to remove source and dist directories if present

mkdir -p ${SRCDIR} ${INSTALLDIR} ${DISTDIR} &>/dev/null
 * 1) setup directories

echo;echo echo --- Downloading all required packages --- echo
 * 1) Get all the required packages

cd ${DISTDIR} aa_grab http://us.php.net/distributions/${PHP5}.tar.gz aa_grab http://mirrors.usc.edu/pub/gnu/libiconv/${LIBICONV}.tar.gz aa_grab http://umn.dl.sourceforge.net/sourceforge/mcrypt/${LIBMCRYPT}.tar.gz aa_grab ftp://xmlsoft.org/libxml2/${LIBXML2}.tar.gz aa_grab ftp://xmlsoft.org/libxml2/${LIBXSLT}.tar.gz aa_grab http://umn.dl.sourceforge.net/sourceforge/mhash/${MHASH}.tar.gz aa_grab http://www.zlib.net/${ZLIB}.tar.gz aa_grab http://curl.haxx.se/download/${CURL}.tar.gz aa_grab http://easynews.dl.sourceforge.net/sourceforge/freetype/${FREETYPE}.tar.gz aa_grab ftp://alpha.gnu.org/pub/gnu/libidn/${LIBIDN}.tar.gz aa_grab ftp://ftp.cac.washington.edu/imap/old/${CCLIENT}.tar.Z wait echo Done.

echo;echo;echo echo --- Unpacking downloaded archives. This process may take several minutes! --- echo
 * 1) Extract the files from the required packages.

cd ${SRCDIR} aa_unpack ${PHP5} aa_unpack ${LIBICONV} aa_unpack ${LIBMCRYPT} aa_unpack ${LIBXML2} aa_unpack ${LIBXSLT} aa_unpack ${MHASH} aa_unpack ${ZLIB} aa_unpack ${CURL} aa_unpack ${LIBIDN} aa_unpack ${CCLIENT} aa_unpack ${FREETYPE} wait

echo -- echo -- Done downloading and unpacking prerequisites -- echo --

exit 0;


 * While it's certainly much smaller, it also doesn't work entirely. I'll have to see if we can incorporate your unpacking idea into the current dev script though. The one reason it doesn't work is due to the fact that not all of the programs are packaged in a tar gzip (.tar.gz) format. Several are bzip2 format and, as with the fallback packages, some don't even have the same format for their fallback package (one is in tar.gz and the other is .bz2). Not a huge issue of course but certainly something that can't be overlooked. Your use of Curl is also quite nice.. perhaps a bit overboard, but we can certainly consider its advantage over wget if you'd like? Mousee 04:30, 17 December 2007 (PST)