Recently at work we needed a way to log successful and failed hits to automatically disable web services for a failover system. There are many ways to implement this but for our application's needs and available tools, only two ideas made sense.

  1. MySQL: store these values on a memory table with delayed insertions.
  2. Memcached: implement Zend_Cache_Backend_Memcached ::increment function

I have seen the mysql solution working great for a couple thousands hits every minute but with our application hitting sometimes 10.000 web service calls every minute we became skeptical really fast. The memcached solution has two major problems: first it's memcaheD you don't know if the value you put in cache will actually make it back and secondly the Zend Memcached adapter needs a patch to implement the increment and a custom method to retrieve the counters. The reason increment has never been officially implemented before in ZF1 is because Zend stores values to cache in a custom way alongside the cache date and lifetime.

$result = @$this->_memcache->set($id, array($data, time(), $lifetime), $flag, $lifetime);

Memcached::increment needs the values to be numerical. I can only speculate why ZF1 dev team decided to save all values as array but anyway. I added the following two methods in Zend\Cache\Backend\Memcached.php

/**
 * Memcached::increment
 *
 * @param string $id cache id
 * @param numerical $offset to increase counter
 * @param numerical $initial value to set counter
 * @param int $extraLifetime
 * @return numerical new items value on success or FALSE on failure.
 */
public function increment($id, $offset = 1, $initial = 0, $specificLifetime = false)
{
	$result = $this->_memcache->increment($id, $offset);
	if (!$result) {
			$lifetime = $this->getLifetime($specificLifetime);
			$this->_memcache->add($id, $initial, false, $lifetime);
			$result = $this->_memcache->increment($id, $offset);
	}
	return $result;
}

/**
 * Custom get to retrieve incremental items
 *
 * @param string $id cache id
 * @return numerical|false cached item
 */
public function getCounterKey($id)
{
	$tmp = $this->_memcache->get($id);
	return is_numeric($tmp) ? $tmp : false;
}

The getCounterKey  is pretty straight forward, we wanted to bypass the default load method that tries to access the data from a custom array. The increment is a little more complex because we wanted a thread safe solution. The memcached::increment and memcached::add used like this, is thread safe.

Unfortunately you can't use these methods directly from the Zend_Cache_Core instance, you need a new patch for that to modify the Core class and all the adapters to add this new capability. It's doable but we were satisfied with just using the following snippet to access the Memcached adapter and the two new methods directly.

$cache = Zend_Registry::get('cache')->getBackend();
$cache->increment($key, 1, 0, 0);
$cache->getCounterKey($key);

The solution works like a charm when memcached is working properly but i would love someday to also test the MySQL idea. I would love to hear what solution other people used in similar problems.

Lithium's Model finders is a great way to avoid repeating the same code blocks over and over. If you google `Lithium finders` you will find many tutorials on this subject but most of the examples don't make much sense (at least to me) since they can be replaced by static methods inside a model class. I have set a simple rule on this matter: methods that don't apply to more than one model class, shouldn't become finders.

To take advantage of finders you must create your own Base model that other models will extend. Below you can see what i use with my two most used finders that i would actually love to see them become official...

class Model extends \lithium\data\Model {
	public static function __init() {
		static::finder('columns', function($self, $params, $chain) {
			$options = &$params['options'];
			$options['return'] = 'resource';
			if ($options['fields'] === null || empty($options['fields'])) {
				$options['fields'] = array($self::meta('key'));
			} elseif (count($options['fields']) > 1) {
				$options['fields'] = array_slice($options['fields'], 0, 1);
			}

			$result = array();
			foreach ($chain->next($self, $params, $chain) as $data) {
				$result[] = $data[0];
			}
			return $result;
		});

		static::finder('pairs', function($self, $params, $chain) {
			$options = &$params['options'];
			$options['return'] = 'resource';
			if ($options['fields'] === null || count($options['fields']) < 2) {
				$key = $self::meta('key');
				$title = $self::meta('title');
				$options['fields'] = array($key, $title);
			} elseif (count($options['fields']) > 2) {
				$options['fields'] = array_slice($options['fields'], 0, 2);
			}

			$result = array();
			foreach ($chain->next($self, $params, $chain) as $data) {
				$result[$data[0]] = $data[1];
			}
			return $result;
		});
	}
}
Read More…

NetBeans allows you to define either global or project specific code formatting options. To apply the following rules globaly navigate to Tools > Options > Editor > Formatting and select PHP language. To define project specific rules right click on your project and go to Properties > Formatting and check the `Use project specific options` and of course select PHP language.

Because the default rules of NetBeans 7.3 are really close to Lithium's coding standarts, we don't need to change many options. Let's begin...


netbeans_lithium_formatting.jpgTabs and Indents

  • Expand Tabs to Spaces: Uncheck
  • Number of Spaces per Indent: 4
  • Tab Size: 4
  • Right Margin: 120
  • Line Wrap: Off
  • Continuation Indentation: 0
  • Array Declaration Indentation: 4

Blank Lines

  • After Function: 0
  • After Open PHP Tag: 0
  • Before Close PHP Tag: 1
Read More…

banner-772x250.png

screenshot-1.pngSummy for wordpress generates excerpts for your posts by applying various algorithms for automatic summarization extraction. It scores your text's sentences, based on extended configuration options, and returns the highest ranked. WP-Summy is based on the Sum+my and was created in an attempt to further develop the Core Library through your feedback.

Important Notes

  • Currently only English & Greek languages are supported.
  • PHP 5.3 is required in order to use this plugin
  • It is tested to work for Wordpress 3.5 up to 3.8
  • Please report if it's compatible with older versions of wordpress

How To Use

  • Write your blog post as you normally do
  • Make sure excerpt and summy blocks are on screen
  • Experiment with all the options and hit Summarize

Installation

  1. Upload the entire folder `summy` to the `/wp-content/plugins/` directory
  2. Activate the plugin through the 'Plugins' menu in WordPress
Read More…

Lithium models are very flexible, schema can be lazy loaded by the first call to Model::schema(), unless it has been manually defined in the model subclass. Below there is an example of a model with predefined schema 

class UserGroups extends \lithium\data\Model
{
	protected $_schema = array(
		'id' => array('type' => 'id', 'length' => 10, 'null' => false, 'default' => null),
		'title' => array('type' => 'string', 'length' => 64, 'null' => false, 'default' => null),
		'description' => array('type' => 'string', 'length' => 255, 'null' => false, 'default' => null),
		'admin' => array('type' => 'integer', 'length' => 1, 'null' => false, 'default' => 0),
		'resources' => array('type' => 'string', 'length' => null, 'null' => false, 'default' => null),
	);

	protected $_meta = array(
		'key' => 'id',
		'locked' => true,
		'source' => 'user_groups'
	);
}

If you leave the schema configuration empty,lithium will retrieve the information directly from the data source. Which is nice until you understand that this extra step will be taken every time the model is initialized, which is a no no. That's why I am going to show you how to cache the model's schema. Firstly we need to set up a connection and the cache storage, here is some basic configuration:

use lithium\data\Connections;
use lithium\storage\Cache;

Cache::config(array(
	'default' => array(
		'adapter' => 'File',
		'strategies' => array('Serializer')))
);

Connections::add('default', array(
	'type' => 'database',
	'adapter' => 'MySql',
	'host' => 'localhost',
	'login' => 'root',
	'password' => '',
	'database' => 'database',
	'encoding' => 'UTF-8'
));

You probably have those two already in your application's bootstrap files, here is the piece of code that does the schema caching

Connections::get('default')->applyFilter('describe', function($self, $params, $chain) {
	if(!Environment::is('production')) {
		return $chain->next($self, $params, $chain);
	}

	$key = 'schema_' . md5(serialize($params['meta']));
	if(!$result = Cache::read('default', $key)) {
		$result = $chain->next($self, $params, $chain);
		Cache::write('default', $key, $result, '+1 month');
	}
	return $result;
});

By using Lithium's filters we can hook into the data source describe method to intercept default behavior. Firstly there is a check to skip caching if we are not in production environment. Then the system checks if schema has  already been in cache by using the model's meta as a unique key. If not, we allow the default procedure to run and we write the result to cache. 

Some notes
Recently Lithium's model and pretty much all data classes have been heavily updated, in many cases breaking backward compatibility. One of them is the model schema, when fully initialized is now an object instead of a simple arrray. The above snippet was written for the latest version in the dev branch but it will probably work for previous versions as well.

The best practice when writing a model is to manually define the schema. If you follow this principle I am not sure the above snippet will improve  much your app's performance. Initialize object vs retrieve object from cache. Maybe with a more advance cache adapter like APC???

Let me know how this worked for you!

FMScout: Twitter Spotlight is another project for fmscout.com i recently finished. The purpose of the plugin is to show status updates from multiple twitter users in a quick and fancy way. The plugin is built to work with the new Twitter Api1.1 rate limits and with the proper configuration it's possible to actually show tweets from a big number of accounts without worrying about getting banned from twitter. Currently fmscout.com is only fetching data from ~15 accounts with ~40 to be the ultimate goal but going tripple digits is possible with longer Cache TTL policy.

tweets0.png tweets1.png

It involved a lot of javascript code, and the js plugin is based on JQuery and JsRender to render all the on the fly html code. The tweets appear in multiple tiles which get flipped randomly every x seconds. All tiles and twitter accounts get the same "spotlight" by shuffling them like a deck of cards instead of randomly picking one every time. My javascript was a little rusty and this project was a lot of fun to build and see in action.

Tweets: Twitter Spotlight

  • PHP: Cotonti Plugin
  • Javascript: JQuery Plugin
  • Site: www.fmscout.com
  • Finished in 4 days after the work request

I have already introduced you to this application, and i already told you about the porting process to Zend Framework 2 but that's not all. Since i finished the first version which only worked for Greek documents, i always wanted to add support for the English language and extend it further more.

So after a few weeks of work and research here it is with a brand new name, Sum+my, which stands for Summarization Methodology Yardstick.

summy_home.png summy_admin_documents.png summy_admin_terms.png  

 

Changelog v2.0

  • Brand New Appearance based on Twitter Bootstrap
  • Ported Application to Zend Framework 2.0
  • Added Support for English Language
  • Added Stemmer Test Page
  • Fixed typos in the Greek Stemmer, improved accuracy a lot!
  • Summaries are now cached, to avoid double posting, the links are by no means permanent!

 summy_admin_home.png summy_stemmer_en.png summy_changelog.png

It's probably the work i am most proud because it's so out of my element, so give it a try you might  actually find it a great tool for everyday tasks and let me know if you have any ideas to improve it further or why not start adding support for more languages. For that you might also want to check the docs of the Sum+my

 

I finished yesterday porting Sum+my to zf2, it will be online after some new features and further development. The process wasn't very hard and you don't have to master zf2 before you begin or even to finish the task. There are already a lot of resources about zf2 and to be able to see code from real life applications from the modules.zendframework.com is a huge bonus. In my case it was a very small application with only 5-6 controllers with a few extra controller plugins, acl + auth + the 15 classes based on Zend\Filter, and it took about 5 hours of work, mostly reading and copy/paste.

First of all performance impressions, out of the box with zero optimization, zf2 is a little friendlier on memory but a little worse on speed. The differences are small but noticeable, i can't give you excact numbers cause i couldn't bother with extensive tests so take my opinion or leave it.

Secondly some tips:

  1. If you are building multi-modular applications avoid doublicate configurations, otherwise you will often be stuck wondering why some changes don't take effect.
  2. The code hasn't changed that much, zf2 hasn't been rewritten from scratch with a few exceptions, so you will often end up wondering what's up with all the fuss about zf2 being hard to start with 
    $error = false;
    $return = $this->params()->fromQuery('return');
    if($this->getRequest()->isPost())
    {
    	$username = $this->params()->fromPost('username');
    	$password = $this->params()->fromPost('password');
    	$return = $this->params()->fromPost('return');
    
    	if($username != '' AND $password != '')
    	{
    		$adapter = $auth->getAdapter();
    		$adapter->setIdentity($username)
    				->setCredential(hash('SHA256', $password));
    
    		$result = $adapter->authenticate();
    		if($result->isValid())
    		{
    			$auth->getStorage()->write($adapter->getResultRowObject(null, 'password'));
    			if($return)
    			{
    				return $this->redirect()->toUrl(base64_decode($return));
    			}
    			return $this->redirect()->toRoute('default', array('controller' => 'index', 'action' => 'index'));
    		}
    	}
    	$error = true;
    }
    

    zf2 vs zf1

    if($this->getRequest()->isPost())
    {
    	$username = $this->_request->getParam('username');
    	$password = $this->_request->getParam('password');
    
    	if($username != '' AND $password != '')
    	{
    		$adapter = new Zend_Auth_Adapter_DbTable(Zend_Db_Table::getDefaultAdapter());
    		$adapter->setTableName('user')
    				->setIdentityColumn('username')
    				->setCredentialColumn('password')
    				->setIdentity($username)
    				->setCredential(hash('SHA256', $password));
    
    		$auth = Zend_Auth::getInstance();
    		$result = $auth->authenticate($adapter);
    		if($result->isValid())
    		{
    			$auth->getStorage()->write($adapter->getResultRowObject(null, 'password'));
    			$returnUri = $this->_getParam('returnUri');
    			$this->_helper->redirector->gotoUrlAndExit($returnUri);
    		}
    	}
    	$this->view->loginFailed = true;
    	$this->view->returnUri = $this->_getParam('returnUri');
    }
  3. The default route you will find in the skeleton app is very basic, so to at least achieve the main function of zf1 default route /controller/action/id use the bellow route, it also supports extra query params, if you call the url helper with route name 'default/query'
    'default' => array(
    	'type' => 'Segment',
    	'options' => array(
    		'route' => '/[:controller[/:action]][/:id]',
    		'constraints' => array(
    			'controller' => '[a-zA-Z][a-zA-Z0-9_-]*',
    			'action' => '[a-zA-Z][a-zA-Z0-9_-]*',
    			'id' => '([0-9]+)?'
    		),
    		'defaults' => array(
    			'__NAMESPACE__' => 'Application\Controller',
    			'controller' => 'Application\Controller\Index',
    			'action' => 'index',
    			'id' => false
    		),
    	),
    	'may_terminate' => true,
    	'child_routes' => array(
    		'query' => array(
    			'type' => 'Query',
    		),
    	),
    ),
  4. If you stepped on a bug there is a good possibility you are not the first one to discover it, check the zf2 github as much as possible.
  5. Use the zf2 classmap generator to make autoloading a hell of a lot faster.

 

That's all for now, i will continue posting about ZF2, my opinion about it has improved a lot since i started porting one of my old zf1 applications and check back maybe i will start posting some very interesting tutorials about Zend Framework 2

 

fmsdb.png

FM Social scouting aka codename FMSDB was a project i did almost 18 months ago for fmscout.com but never went live for legal reasons. All the difficulties and delays are finally over and it will go public hopefully this week. FMSDB is practically a database for football (soccer) players and clubs based on the video game Football Manager. Except all the personal information and game statistics every profile page has comments and ratings support hence Social Scouting and photo galleries as well. Until the project goes officially live that's all i can say but here is a couple of screenshots. The project is live you can check it online www.fmscout.com/players.html

 fmsdb_player_search.png fmsdb_player_profile.png

The application is build as a module for Cotonti cms, it involved mostly php/mysql code and data mining which is something i always like. Originally image galleries were hosted by imageshack but since last year they changed their policy and galleries had to be rewritten from scratch, talk about snafu right ? that's one piece of code i won't be reusing any time soon. ;p The past few weeks we also did the beta testing and small fine-tuning a year overdue.

Day number two of porting this application to zend framework 2. I only had a couple of hours today but still i have a couple of things i actually want to talk about. I keep encountering things that are not that much different from ZF1 like Zend\Navigation or Zend\Paginator. I was quite surprised i got them working in seconds.

In the app i have a paginated list of documents, and below is how the code changed:

Zend Framework 1

$db = Zend_Db_Table_Abstract::getDefaultAdapter();
$select = $db->select()->from('document', $orders)->order(array("$order $way"));
$paginator = Zend_Paginator::factory($select);
$paginator->setDefaultItemCountPerPage($perpage);
$paginator->setCurrentPageNumber($this->_getParam('page'));

Zend Framework 2

$db = $this->getServiceLocator()->get('db');
$select = new Select('document');
$select->columns($orders)->order("$order $way");
$paginator = new Paginator(new DbSelect($select, $db));
$paginator->setDefaultItemCountPerPage($perpage);
$paginator->setCurrentPageNumber($page);

 

 Which if you exclude all the Use Statements you have to add on top it's not bad, i actually think it pretty much the same. It would be nice to be able to get the Select object from the adapter but it's not a big deal. The second thing of the day was how actively the ZF2 is developed. I found a couple bugs today working with v2.0.3. The most important was with the Zend\Mvc\Router\Http\Query which was adding the controller, action and namespace parameters to the query string, instead of only adding the extra parameters it was receiving. But guess what download the latest version from github and voila bugs are gone. (Lithium are you there... ;p). That really made my day, i make it a habit not following popular trends or rooting for the underdogs, see my history. I started learning php with LDU cms probably the most unkown cms of all and now choosing Lithium for Komposta Core instead of one of the most popular frameworks like Zend or Symphone or CakePHP for pete sake. So it felt great to go from the disappointing moment  of finding a bug to see that i was fixed in a minute. People are whining how much ZF2 changed and that it got a bit Java-ish but today at least, in my mind ZF2 will succeed because of the active developers and contributors.