PHP Fog - Free 6 months.. what next, and how much?

I was very disappointed when I asked PHP-Fog how much it would cost after 6 months using their free package. They move you to a "Micro Instance" package which turns out to be $29 a month charge.

I can't see any cost benefit here to moving to a so-called 'cloud' environment, if its a MINIMUM 29 dollars a month after 6 months. This means for the vast for the majority of websites and web apps its simply just not worth it. The true cloud environments such as Google's App Engine (which does not have any PHP support unfortunately) charge depending on CPU and data-store usage. A usage based charge basically, not a fixed monthly one, which is what cloud hosting is all about.

PHP Fog seems to have just got on the cloud bandwagon, and what you're basically getting is a 'pretend' cloud hosting environment that is WAY more expensive than your traditional PHP hosting companies.

A big disappointment to the PHP community.

Datastore inremental backup script for App Engine

I'm a developer using App Engine as a back-end for my Android App - Chesspresso. I love App Engine and it has come a long way since it was incubated a few years back and for Chesspresso it has been a lifesaver in terms of scalability. But one critical thing thats missing currently though is datastore backup.

Here is a python script that I use to backup my app datastore in a cron job on remote servers. It creates incremental backups of the GAE datastore zipped up with logs, and it'll mail you if the shit hits the fan. Its worked for me nicely so far. You'll have to go through it with a fine tooth comb as there is a lot of variables to tweak for your own uses.

BTW, You'll have you install the remote_api servlet in your app for this to work...

import os
import shutil
import subprocess
import sys
import time
import zipfile
import glob
from zipfile import ZipFile
import traceback
import string
import smtplib
import StringIO

# Folders
FOLDER_ROOT = "/keep-everything-in-this-folder";
FOLDER_TEMP = FOLDER_ROOT + "/tmp";
FOLDER_BACKUPS = FOLDER_ROOT + "/backup";
FOLDER_GAE = FOLDER_ROOT + "/gaep";
SERVER_SMTP = "localhost";

# Other
GAE_APPNAME = "your-gae-app-name";
GAE_URL_API = "/remote_api";
FOLDER_NAME_TEMP = "gaebak";
FILE_NAME_APPCFG = "appcfg.py";
FILE_NAME_INPUT = "input.txt";
FILE_NAME_ARCHIVE = time.strftime("%y%m%d-%H%M%S") + ".zip";
ERROR_EMAIL = "your-email";

# max number of archives before deletion
MAX_ARCHIVES = 50;

# Arguments
ARG_URL = "--url=http://" + GAE_APPNAME + ".appspot.com" + GAE_URL_API;
ARG_APP = "--application=" + GAE_APPNAME;
ARG_LOG = "--log_file=log-";
ARG_PASSIN = "--passin";

# Full Paths
F_TEMP = os.path.join(FOLDER_TEMP, FOLDER_NAME_TEMP);
F_APPCFG = os.path.join(FOLDER_GAE, FILE_NAME_APPCFG);
F_BACKUP = os.path.join(FOLDER_BACKUPS, GAE_APPNAME);
F_ARCHIVE = os.path.join(F_BACKUP, FILE_NAME_ARCHIVE);
F_INPUT = os.path.join(FOLDER_ROOT, FILE_NAME_INPUT);

def removeTempFolder():
    if os.path.exists(F_TEMP):
        shutil.rmtree(F_TEMP);
    return;

def createTempFolder():
    if os.path.exists(F_TEMP) is False:
        os.mkdir(F_TEMP);
    return;

def createBackupFolder():
    if os.path.exists(F_BACKUP) is False:
        os.mkdir(F_BACKUP);
    return;

def call(type, extra):
    p = subprocess.call([sys.executable, F_APPCFG, ARG_PASSIN, ARG_URL, ARG_APP, extra, type], stdin=file(F_INPUT));
    return;

def init():
    removeTempFolder();
    createTempFolder();
    createBackupFolder();
    os.chdir(F_TEMP);
    return;

def backup():
    call("--filename=db.dump", "download_data");
    return;

def archiveBackup():
    zip = zipfile.ZipFile(F_ARCHIVE, "w", zipfile.ZIP_DEFLATED);
    
    for name in glob.glob1(F_TEMP, "*"):
        zip.write(os.path.join(F_TEMP, name), name);
        
    zip.close();
    return;

def pruneArchives():
    files = glob.glob1(F_BACKUP, "*.zip");
    files.sort(reverse=True);
    num = len(files);
    
    for i in range(MAX_ARCHIVES, num):
        os.remove(os.path.join(F_BACKUP, files[i]));
    
    return;

def sendMail(trace):
    emailFrom = ERROR_EMAIL;
    emailTo = [ ERROR_EMAIL ];
    emailSubject = "[GAE Backup] ERROR occurred while running backup script: (" + F_ARCHIVE + ")";
    emailBody = trace;

    body = string.join(("From: %s" % emailFrom, "To: %s" % emailTo, "Subject: %s" % emailSubject, "", emailBody), "\r\n");
    server = smtplib.SMTP(SERVER_SMTP);
    server.sendmail(emailFrom, emailTo, body);
    server.quit();
    return;

# Execute
try:
    init();
    backup();
    archiveBackup();
    pruneArchives();
except Exception:
    sendMail(traceback.format_exc());
    traceback.print_exc(file=sys.stdout);

You'll also see 'input.txt' mentioned up at the top there. That is a file which stores your user/pass for authorization. Its simple and it looks like this:

mygmailaddresonthefirstline@gmail.com
mygmailpasswordonthe2ndline

Sorry all, probably not the most elegant of technical blog posts, but hopefully it'll come in handy for someone else. As far as I know this is compatible with Linux, but not tried on any other OS.

Activity Launch Modes - A simple explanation in a table.

An Android Activity launch mode dictates to which Task an Activity is created and how it is instantiated. Its a tiny bit confusing to figure out exactly what launch mode you need from the official documentation so I've created a small table to help.

First of all here is a brief definition of some of the terms:

Task - A "stack of Activities" in your application. If an Activity is sent to the background (by pressing the HOME key whilst viewing it, for example) then the whole Task (and all the Activities inside it) will be sent back as well. If the user then clicks on your application, the task (and the order of its activities) come forward.

Root of a task - The first Activity in a Task. There is always one of these in a Task.

Essentials at Poundland in Colwyn Bay!

This has nothing to with Android, or anything techie but I thouoght it was a quite funny.. 

My sister took this photo this back in our home town of Colwyn Bay in Poundland shop. Apparently 2 pregnancy kits were a pound - Now that's value for money!

HttpClient and ConnectionPoolTimeoutException

I've recently had a bit of trouble with my application's DefaultHttpClient instance throwing a ConnectionPoolTimeoutException quite consistently after a number of requests to the same location. (See bottom of the entry for a stacktrace).

There doesn't seem to be anything regarding this particular stacktrace in any of the Android developer forums up to now but coincidently Axis2 framework users (Also using Apache HttpClient) reported the same exception occuring after 3 requests to the same location, which is exactly what I experienced.

Here was my instance creation code for the HttpClient:

HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);

SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));

// Where ClientConfig.P_DEFAULT_CONN_TIMEOUT = 15 seconds
HttpConnectionParams.setConnectionTimeout(params, ClientConfig.P_DEFAULT_CONN_TIMEOUT);
HttpConnectionParams.setSoTimeout(params, ClientConfig.P_DEFAULT_CONN_TIMEOUT);
ConnManagerParams.setTimeout(params, ClientConfig.P_DEFAULT_CONN_TIMEOUT);

httpClient = new DefaultHttpClient(new ThreadSafeClientConnManager(params, schemeRegistry), params);

If you're using TheradSafeClientConnManager be aware that there is a "Max connections per route" property available which is pretty well hidden, and again isn't mentioned in any of the example code I came across. Setting this to a higher value got rid of my issue.

...
ConnManagerParams.setMaxConnectionsPerRoute(params, new ConnPerRoute() {
    @Override
    public int getMaxForRoute(HttpRoute httproute)
    {
        return 10;
    }
});
...

The stacktrace:

fetch() sending failed to url http://192.168.0.45/blah
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection
    at org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByRoute.java:353)
    at org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRoute.java:238)
    at org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(ThreadSafeClientConnManager.java:175)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:325)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:555)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:465)
    at chesspresso.client.android.net.HttpCall.fetch(HttpCall.java:114)
    at chesspresso.client.android.net.HttpCall.fetch(HttpCall.java:134)
    at chesspresso.client.android.net.HttpCall.post(HttpCall.java:107)
    at chesspresso.client.android.net.ServerCallGoogle.loginGoogle(ServerCallGoogle.java:30)
    at chesspresso.client.android.activity.support.ActivityProcessesGoogle$LoginGoogleTask.doInBackground(ActivityProcessesGoogle.java:72)
    at chesspresso.client.android.activity.support.ActivityProcessesGoogle$LoginGoogleTask.doInBackground(ActivityProcessesGoogle.java:1)
    at android.os.AsyncTask$2.call(AsyncTask.java:185)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
    at java.util.concurrent.FutureTask.run(FutureTask.java:137)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:561)
    at java.lang.Thread.run(Thread.java:1096)

Apache HttpClient is obviously used quite a lot by many apps yet it's configurable parameters haven't been explained in android examples as well as they should be I would say.

Samsung Galaxy Tab - Bit of a Gimmick

I don't normally purchase cutting edge technology on their day of release, but this time around the Galaxy Tab caught my attention. Over the last few days I followed internet reviews, opinions and videos of the release of this thing and after a while I convinced myself that I needed to buy one. I definitely needed a Froyo based device of some sorts as I've been using a 1.5 device for a long time as my main phone and that was the only piece of hardware I had for development. For development a decent handset is essential as of course you need to check your work on the real thing every now and again and the latest release emulators can be painfully slow especially in debug mode, so I went to the Carphone Warehouse yesterday on the day of release and purchased one for 499 GBP.

Probably the most expensive gadget I will ever buy, but the best price I could find - A 1GB data only 1-month rolling contract with TalkTalk (and a free month) that I can cancel any time - which I probably will in the next few days. This for 499GBP.

After 24 hours with this thing now I'm a bit surprised at the large number of positive reviews this device has received... Its main flaw is that it doesn't really fill in a space that needs to be filled! It's an oversize mobile phone that simply can't replace your laptop or netbook for any real purpose.

I use Gmail and Google Reader a lot for example, and I was thinking this would be a great device just to read my mail and browse my feeds. Replying to email with the on-screen keyboard is annoying and the mobile gmail client is no where near as good as what you'd receive on your laptop browser. The same with Google Reader. So many features that aren't available that you just end up using your laptop/netbook.

I also thought this thing would be great for Skype. No it's not. Again down to the actual Skype application for Android because it doesn't seem to support video calling at the moment and again there are far too many features that are missing from what you could be installing on your PC. Again, another reason to ditch the tablet and pick up your laptop instead.

Of course this thing won't replace your mobile phone either because it is far too big to carry around in your pocket. I was thinking of taking it to the gym this morning and to try a particular excercise / rep application and maybe to listen to some podcasts, but really having this thing AND my mobile is just silly. It works as a mobile of course and supports Bluetooth etc. but it's far too big to bring up to your ear as a mobile phone.

At a price of usually 530 GBP it is incredibly expensive for a bit of a gimmick really. I guarantee that more often than not it won't allow you to do what you want as well as your current laptop and mobile phone. You'll also look like a bit of an idiot if someone phones you on it.

And another little criticism before I go! I have a spot in the house from which I use my laptop quite often and I usually get 2 bars on my Windows 7 status bar icon. The Tab can't find my access point from the same spot! I didn't notice any review as yet which also commented on the quality of the wifi but in my opinion it's definitely not as good as my laptop wifi interface.

Sorry, but I'm not a convinced with this one. A bad impulse purchase!

GAE and Log4j - Getting them to work a bit better together

The Problem
App Engine supports Log4j, and you'll be able to see your log4j statements come through either as a stdout or stderr message. The trouble that I've had is that if you want to make full use of the nice log highlighting GAE supports in its console then all stdout / stderr messages will come through as a warning.

A good portion of Java frameworks use Log4j as their logging framework. I'm using Apache Wicket as my front-end framework for example, and when user has hit upon a bug on a page, I'd like to know about it as an error and not just a warning.

A Solution
Here's a Log4j Appender which I've written that creates JUL log statements based on original Log4j log events. A Log4j -> JUL bridge if you like, passing on the original severity level to JUL. It's pretty basic, but it works for me!

Here's the code for the Appender itself:

/**
 * 
 * @author Eurig Jones
 */
public class GAELogAppender extends AppenderSkeleton
{
    private static final Logger L = Logger.getLogger("log4j");

    @Override
    protected void append(LoggingEvent event)
    {
        StringBuilder b = new StringBuilder();
        Level level = event.getLevel();
        String s[] = event.getThrowableStrRep();

        b.append(event.getMessage().toString());

        if (s != null)
            for (String line : s)
                b.append(line).append('\n');

        if (level.equals(Level.FATAL))
            FATAL(b.toString());
        if (level.equals(Level.ERROR))
           ERROR(b.toString());
        if (level.equals(Level.WARN))
           WARN(b.toString());
        if (level.equals(Level.INFO))
           INFO(b.toString());
        if (level.equals(Level.DEBUG))
           DEBUG(b.toString());
        if (level.equals(Level.TRACE))
           TRACE(b.toString());
    }

    private void FATAL(String msg)
    {
        L.severe(msg);
    }

    private void ERROR(String msg)
    {
        L.severe(msg);
    }

    private void WARN(String msg)
    {
        L.warning(msg);
    }

    private void INFO(String msg)
    {
        L.info(msg);
    }

    private void DEBUG(String msg)
    {
        L.fine(msg);
    }

    private void TRACE(String msg)
    {
        L.finest(msg);
    }

    @Override
    public boolean requiresLayout()
    {
        return false;
    }

    @Override
    public void close()
    {
    }
}

Now all we need to do is setup the log4j.properties and logging.properties configurations correctly to make it work. Here is a snippet of my log4j.properties:

#log4j.appender.A1=org.apache.log4j.ConsoleAppender
#log4j.appender.A1.layout=org.apache.log4j.PatternLayout
#log4j.appender.A1.layout.ConversionPattern=%d ...
#log4j.appender.A1=org.apache.log4j.jul.JULAppender

log4j.appender.A1=chesspresso.server.log.GAELogAppender

log4j.logger=ERROR, A1

...
log4j.logger.org.apache.wicket=WARN, A1
...

Above shows that I've replaced the original ConsoleAppender - the default appender which GAE uses, with the new GAELogAppender. This allowing all Log4j statements to go through this instead of the console.

Now the GAE logging.properties file...

.level = INFO
chesspresso.server.log.GAELogAppender=FINEST
...

As you can see, I've added the GAELogAppender here to allow all severities to be filtered through. This means that anything that hits our new appender will reach the GAE logging system, as long as our log4j.properties will allow it to filter through.

I'm still a bit unsure of how exactly GAE Maps JUL severities in it's frontend. In JUL you have FINEST, FINER, INFO, WARNING, SEVERE but the front-end seems to display these differently. It has an ERROR and CRITICAL as well as a WARNING. SEVERE seems to be flagged as an ERROR, but how can you flag something as CRITICAL? Maybe it's in some documentation I've probably scanned through :-)

This is a quick implementation of this Appender for me and so far it works, but I'd love to hear some feedback about what it doesn't do (or doesn't do properly) as I've not used it in anger.