Recently at work we needed a way to log successful and failed hits to automatically disable web services for a failover system. There are many ways to implement this but for our application's needs and available tools, only two ideas made sense.

  1. MySQL: store these values on a memory table with delayed insertions.
  2. Memcached: implement Zend_Cache_Backend_Memcached ::increment function

I have seen the mysql solution working great for a couple thousands hits every minute but with our application hitting sometimes 10.000 web service calls every minute we became skeptical really fast. The memcached solution has two major problems: first it's memcaheD you don't know if the value you put in cache will actually make it back and secondly the Zend Memcached adapter needs a patch to implement the increment and a custom method to retrieve the counters. The reason increment has never been officially implemented before in ZF1 is because Zend stores values to cache in a custom way alongside the cache date and lifetime.

$result = @$this->_memcache->set($id, array($data, time(), $lifetime), $flag, $lifetime);

Memcached::increment needs the values to be numerical. I can only speculate why ZF1 dev team decided to save all values as array but anyway. I added the following two methods in Zend\Cache\Backend\Memcached.php

/**
 * Memcached::increment
 *
 * @param string $id cache id
 * @param numerical $offset to increase counter
 * @param numerical $initial value to set counter
 * @param int $extraLifetime
 * @return numerical new items value on success or FALSE on failure.
 */
public function increment($id, $offset = 1, $initial = 0, $specificLifetime = false)
{
	$result = $this->_memcache->increment($id, $offset);
	if (!$result) {
			$lifetime = $this->getLifetime($specificLifetime);
			$this->_memcache->add($id, $initial, false, $lifetime);
			$result = $this->_memcache->increment($id, $offset);
	}
	return $result;
}

/**
 * Custom get to retrieve incremental items
 *
 * @param string $id cache id
 * @return numerical|false cached item
 */
public function getCounterKey($id)
{
	$tmp = $this->_memcache->get($id);
	return is_numeric($tmp) ? $tmp : false;
}

The getCounterKey  is pretty straight forward, we wanted to bypass the default load method that tries to access the data from a custom array. The increment is a little more complex because we wanted a thread safe solution. The memcached::increment and memcached::add used like this, is thread safe.

Unfortunately you can't use these methods directly from the Zend_Cache_Core instance, you need a new patch for that to modify the Core class and all the adapters to add this new capability. It's doable but we were satisfied with just using the following snippet to access the Memcached adapter and the two new methods directly.

$cache = Zend_Registry::get('cache')->getBackend();
$cache->increment($key, 1, 0, 0);
$cache->getCounterKey($key);

The solution works like a charm when memcached is working properly but i would love someday to also test the MySQL idea. I would love to hear what solution other people used in similar problems.

I finished yesterday porting Sum+my to zf2, it will be online after some new features and further development. The process wasn't very hard and you don't have to master zf2 before you begin or even to finish the task. There are already a lot of resources about zf2 and to be able to see code from real life applications from the modules.zendframework.com is a huge bonus. In my case it was a very small application with only 5-6 controllers with a few extra controller plugins, acl + auth + the 15 classes based on Zend\Filter, and it took about 5 hours of work, mostly reading and copy/paste.

First of all performance impressions, out of the box with zero optimization, zf2 is a little friendlier on memory but a little worse on speed. The differences are small but noticeable, i can't give you excact numbers cause i couldn't bother with extensive tests so take my opinion or leave it.

Secondly some tips:

  1. If you are building multi-modular applications avoid doublicate configurations, otherwise you will often be stuck wondering why some changes don't take effect.
  2. The code hasn't changed that much, zf2 hasn't been rewritten from scratch with a few exceptions, so you will often end up wondering what's up with all the fuss about zf2 being hard to start with 
    $error = false;
    $return = $this->params()->fromQuery('return');
    if($this->getRequest()->isPost())
    {
    	$username = $this->params()->fromPost('username');
    	$password = $this->params()->fromPost('password');
    	$return = $this->params()->fromPost('return');
    
    	if($username != '' AND $password != '')
    	{
    		$adapter = $auth->getAdapter();
    		$adapter->setIdentity($username)
    				->setCredential(hash('SHA256', $password));
    
    		$result = $adapter->authenticate();
    		if($result->isValid())
    		{
    			$auth->getStorage()->write($adapter->getResultRowObject(null, 'password'));
    			if($return)
    			{
    				return $this->redirect()->toUrl(base64_decode($return));
    			}
    			return $this->redirect()->toRoute('default', array('controller' => 'index', 'action' => 'index'));
    		}
    	}
    	$error = true;
    }
    

    zf2 vs zf1

    if($this->getRequest()->isPost())
    {
    	$username = $this->_request->getParam('username');
    	$password = $this->_request->getParam('password');
    
    	if($username != '' AND $password != '')
    	{
    		$adapter = new Zend_Auth_Adapter_DbTable(Zend_Db_Table::getDefaultAdapter());
    		$adapter->setTableName('user')
    				->setIdentityColumn('username')
    				->setCredentialColumn('password')
    				->setIdentity($username)
    				->setCredential(hash('SHA256', $password));
    
    		$auth = Zend_Auth::getInstance();
    		$result = $auth->authenticate($adapter);
    		if($result->isValid())
    		{
    			$auth->getStorage()->write($adapter->getResultRowObject(null, 'password'));
    			$returnUri = $this->_getParam('returnUri');
    			$this->_helper->redirector->gotoUrlAndExit($returnUri);
    		}
    	}
    	$this->view->loginFailed = true;
    	$this->view->returnUri = $this->_getParam('returnUri');
    }
  3. The default route you will find in the skeleton app is very basic, so to at least achieve the main function of zf1 default route /controller/action/id use the bellow route, it also supports extra query params, if you call the url helper with route name 'default/query'
    'default' => array(
    	'type' => 'Segment',
    	'options' => array(
    		'route' => '/[:controller[/:action]][/:id]',
    		'constraints' => array(
    			'controller' => '[a-zA-Z][a-zA-Z0-9_-]*',
    			'action' => '[a-zA-Z][a-zA-Z0-9_-]*',
    			'id' => '([0-9]+)?'
    		),
    		'defaults' => array(
    			'__NAMESPACE__' => 'Application\Controller',
    			'controller' => 'Application\Controller\Index',
    			'action' => 'index',
    			'id' => false
    		),
    	),
    	'may_terminate' => true,
    	'child_routes' => array(
    		'query' => array(
    			'type' => 'Query',
    		),
    	),
    ),
  4. If you stepped on a bug there is a good possibility you are not the first one to discover it, check the zf2 github as much as possible.
  5. Use the zf2 classmap generator to make autoloading a hell of a lot faster.

 

That's all for now, i will continue posting about ZF2, my opinion about it has improved a lot since i started porting one of my old zf1 applications and check back maybe i will start posting some very interesting tutorials about Zend Framework 2

 

Day number two of porting this application to zend framework 2. I only had a couple of hours today but still i have a couple of things i actually want to talk about. I keep encountering things that are not that much different from ZF1 like Zend\Navigation or Zend\Paginator. I was quite surprised i got them working in seconds.

In the app i have a paginated list of documents, and below is how the code changed:

Zend Framework 1

$db = Zend_Db_Table_Abstract::getDefaultAdapter();
$select = $db->select()->from('document', $orders)->order(array("$order $way"));
$paginator = Zend_Paginator::factory($select);
$paginator->setDefaultItemCountPerPage($perpage);
$paginator->setCurrentPageNumber($this->_getParam('page'));

Zend Framework 2

$db = $this->getServiceLocator()->get('db');
$select = new Select('document');
$select->columns($orders)->order("$order $way");
$paginator = new Paginator(new DbSelect($select, $db));
$paginator->setDefaultItemCountPerPage($perpage);
$paginator->setCurrentPageNumber($page);

 

 Which if you exclude all the Use Statements you have to add on top it's not bad, i actually think it pretty much the same. It would be nice to be able to get the Select object from the adapter but it's not a big deal. The second thing of the day was how actively the ZF2 is developed. I found a couple bugs today working with v2.0.3. The most important was with the Zend\Mvc\Router\Http\Query which was adding the controller, action and namespace parameters to the query string, instead of only adding the extra parameters it was receiving. But guess what download the latest version from github and voila bugs are gone. (Lithium are you there... ;p). That really made my day, i make it a habit not following popular trends or rooting for the underdogs, see my history. I started learning php with LDU cms probably the most unkown cms of all and now choosing Lithium for Komposta Core instead of one of the most popular frameworks like Zend or Symphone or CakePHP for pete sake. So it felt great to go from the disappointing moment  of finding a bug to see that i was fixed in a minute. People are whining how much ZF2 changed and that it got a bit Java-ish but today at least, in my mind ZF2 will succeed because of the active developers and contributors.

zf2.pngI 've been following zf2 from the very first beta versions mainly reading presentations, tutorials and occasionally i browsed the code especially the skeleton application to see how the project progressed. So far i wasn't very impressed with it, the common view as i understand it is that it's not quite there yet and that it needs to be refined more before people really start to jump in and port their old applications to the new framework.

Today thought i started porting my BA thesis application from ZF1 to ZF2 as i plan to extend it  and add new features. The application consists of 5 controllers, a few plugins/helpers and the main engine that produces automatic summarization for Greek language which is a bunch of filters also written in ZF1. Beside that there are Users and Administration systems in place to easily manage the internal data.

I actually managed to accomplice more than i thought for day one. I got the front-end controllers and the summarization engine working with ZF2 quite easily. But still some things really bothered me. The most annoying thing of the day is the new Zend\Db it's obvious that they tried to enforce the Model part of the new MVC concept and it's not there not by a mile not when you have to inject every custom model with the TableGetAway which has to also be injected with the Db\Adapter.  

class Module
{
    // getAutoloaderConfig() and getConfig() methods here

    // Add this method:
    public function getServiceConfig()
    {
        return array(
            'factories' => array(
                'Album\Model\AlbumTable' =>  function($sm) {
                    $tableGateway = $sm->get('AlbumTableGateway');
                    $table = new AlbumTable($tableGateway);
                    return $table;
                },
                'AlbumTableGateway' => function ($sm) {
                    $dbAdapter = $sm->get('Zend\Db\Adapter\Adapter');
                    $resultSetPrototype = new ResultSet();
                    $resultSetPrototype->setArrayObjectPrototype(new Album());
                    return new TableGateway('album', $dbAdapter, null, $resultSetPrototype);
                },
            ),
        );
    }
}

I am pretty sure there is a smarter way to go that people already use in their modules but that official example is a joke. In my case there are about 4 mysql tables and in the current code base so i didn't bother with models since the system is quite predictable about future features and queries so i prefered the direct approach, here is an example of ZF1 vs ZF2 for running raw queries.

 Zend Framework 1.11

$db = Zend_Db_Table::getDefaultAdapter();
$docu = $db->fetchOne("SELECT COUNT(*) as unprocessed FROM document WHERE processed = 0");

Zend Framework 2.0.3

$db = $this->getServiceLocator()->get('db');
list($docu) = $db->query("SELECT COUNT(*) as total FROM document WHERE processed = 0")->execute()->current();

So my first day wasn't very positive. It was easy to port basic stuff and start working but on the other hand i am not seeing many advantages so far. Keep in touch as i will post daily about my experience with zf2.

State Scholarships Foundation (IKY in greek) is the official organization who manages scholarships in Greece and once again i was asked to develop a web platform to manage applications for scholarships. If you read my previous post i had already worked on a similar project for a different scholarships program, but for some reason it wasn't organized and managed my IKY but by a sub-division inside the Ministry of Education. Till this day i haven't figured out why. Greece the country of amazing and bizzare things ;p

Thankfully IKY knew how to run things and it was so much easier to work with them. The web platform beside the applications it had to also manage the evaluation process. The sytem had to manage evaluators who were independent professors from all over the world and blind assing applications to evaluators with the proper scientific background. Huge project and probably my best work from last year. Time for info and pictures

  • Powered By Zend Framework
  • PHP/MySql/Html/Css/Jquery
  • Completion Time: 6 weeks plus 8 weeks for tech support for applicants and evaluators
  • August - November 2011

ikyhome.png  ikyappbefore.png ikyapp1.png ikyapp2.png

 

The website was hosted here http://apps.gov.gr/minedu/iky_scholarships, unfortunately this year they didn't use my application. Back in February i got an email from my old partner in Greek Ministry of Education asking for help. Unfortunately i had no time to spare and my military obligations were coming up and i kinda ignored him. For some reason all the programmers in Minedu and IKY didn't do PHP. Most of them were java and asp programmers. My guess is that they didn't found someone who could manage my application and decided to either make a new platform from scratch or and most likely gave the job to a private company. They were going to do the same last year as well, their budget allowed it, but as i later learnt they couldn't find a company to finish the project in such a short time.

 

 

 ikyapp3.png ikyappafter.png ikyadminhome.png ikyadminexport.png

I am really regretting not responding to my old partner, now i am broke, unemployed and too embarrassed to call them back asking if there is any opening for a web programmer.  Hopefully something will come up soon.

 ikyadminappsmanage.png ikyadminapps.pngikyevaluator.png ikyevaluatorevaluate.png

High School students in Greece are admitted to public Universities based on their final exams during their senior year. They are submitting forms with their information and a selection of schools they want to attend. Last year the ministry of education of Greece decided to simplify the applications procedure by making it web only. Two teams of programmers were formed one managed the web platform for students who finished high school in Greece and another for Greek or foreign students who finished high school abroad.

I was part of the second team which consisted of only two people. I was in charged of building the web platforms and Athanasios Rouskas was something like the project manager bringing me all the specifications and dealing with other departments delays and bureaucracy.

Below you can find screenshots from http://mixanografiko-eksoterikou.opengov.gr/ where the application is hosted.

selectSchools.png review.png massEmail.png main.png form.png adminMain.png adminApps.png 

 

The web platform was used again this year, with no updates.  The second application, which had minor changes to meet the state criteria for foreign students, was actually completed by Rouskas who had never seen php code before but he was so damn eager to learn and he learnt. That application was also used this year here http://mixanografiko-alodapon.opengov.gr  

Edit: 15/8/2013
I am actually very proud to see that both platforms are online this year again, 3 years and counting (
2011-2013)

The application was build with Zend Framework/MySql/Jquery. Below you can also find the sample documents the system creates with Zend_Pdf for the users to verify that everything went fine. 

Read about the updated version Sum+My here.

Automatic Summarization for the Greek language is the title of my undergraduate thesis i completed almost a year ago. Because of the obvious difficulties i choose to use for the first time a PHP Framework. I wanted to focus on the task at hand without worrying about the basics involving a web platform. I could have gone with a cms like joomla or drupal but at the time i found Zend Framework 1.xx.x to be so much better for the job, because of the excellent documentation and online sources.


Automatic summarization is a vague subject in the science of natural language processing. There are two main methodologies you can follow, extraction and abstraction. The second is the hardest and involves computer learning techniques (AI). Basically the machine has to learn how to produce the summaries, pretty much like a human. The extraction model (shallow) is based on maths and doesn't really create sentences from scratch, all the sentences come directly from the original document without any alternation.

For an undergraduate thesis working on the abstraction model is a little too much and honestly a semester is not enough time, so i based the application on shallow methods. In an attempt to not reproduce the few source out there i tried to use and combine as many algorithms as possible. The system produces for each sentence three scores, terms, position and keywords which can be used with different weights to evaluate easy score.


For the Terms score the user can choose between:

  1. TF-ISF (Term Frequency - Inverse Sentence Frequency) 
  2. TF-IDF (Term Frequency - Inverse Document Frequency)
  3. TF-RIDF (Term Frequency - Residual Inverse Document Frequency)

Are you still here reader ? Ok stay with me i will be quick. Basically every method produces a score for each word based either:

  1. On all the words inside the document
  2. On all the words inside a collection of documents
  3. 2 + Poisson Distribution Model



For the Position score the user can choose between:

  1. Baxendale's research
  2. News article

Baxendale was a researcher who concluded that in 85% of the paragraphs the topic sentence came as the first one and in 7% of paragraphs the last sentence was the topic sentence. Thus, a naive but fairly accurate way to select a topic sentence would be to choose one of these two. The News Articles algorithm basically scores sentences dynamically and clearly favours the first sentences of the first paragraphs.

The Keywords score is actually a cheat, by providing keywords the system can scores sentences that contain them higher than others. This way to system finds the key sentences easier instead of guessing like the above scoring methods. Additionally the user can select to set a words per sentence threshold, so the system can ignore too big or too small sentences. The system also uses stop words lists to ignore common terms as well a greek language stemming algorithm to group words. 

The application is hosted here http://thesis.t3-design.com/, soon to be hosted on this domain. If you are not Greek you will probably don't understand much but hey it was my first Zend Application and i am really proud about it and please take a look at the credits section, i couldn't have done it without them.

You can download my paper here, once again it's in Greek but don't worry if you want the gist of it my supervisor teachers published a scientific paper based on my work, you can find it  here.

Time for some pictures (new files system in place ;p)

 

The idea of building a cmf came to me 2 years ago, at the time i wasn't into php frameworks, i intended to build it from scratch and i came very close to a release candidate a year ago but my pet project had to be put on hold. I started work at Greek Ministry of Education and i was involved on a lot of web projects and to speed up the procedure i choose Zend Framework as the industry standard choice. That's when i fall in love with php frameworks. Zend 1.11.x was easy to work with, it didn't bound the programmer to follow a strict path and had (probably still has) a huge variety of libraries to accommodate any programmers need. Another big factor on choosing Zend was the excellent documentation and on-line sources.

Six weeks ago when i decided to restart my pet project i had to choose between continuing with my php framework or go Zend. Unfortunately my php framework had to die young because i wanted to have something ready for production this year. Zend framework 2.0 was well under the way but at the time only beta versions had surfaced. I have to admit that after peaking at the source code of ZF2 i wasn't impressed but i won't comment on it until i have the time to check the stable version that was released last week.  

After that i was on a php framework hunt and limited the options pool to Symphony2, Kohana and Lithium. I quickly eliminated Symphony because it looked like it has a very steep learning curve. Kohana, although i promise to keep in touch, is lacking in cutting edge PHP features. After quick tests i was impressed with Lithium. It's relatively small and fast, it doesn't impose stupid restrictions to the programmers and has a better than good in-house Data Model solution. Of course it has some grey areas but because it's so flexible you can bypass any problem or bug easily.

But i have to admit the number one reason for my choice was the news about Lithium getting sponsored by Engine Yards. Honestly Lithium development has slowed down a lot this year and until the news from Engine Yards i wasn't sure if Lithium was dead or alive. Documentation is sketchy at best, online sources are limited and git support is, well hrm... If you are new to php go with a framework with good documentation and community like CakePHP, otherwise be prepared for a lot of code reading or stay tuned, lithium tutorials are on the way...

Hello my name is Chris i am 26 years old, i am a web developer with almost ten years of experience in php and mysql. This site is my personal blog/portfolio. In the past i developed plugins, templates for applications like LDU/Seditio (R.I.P), wordpress, joomla, phpbb, vbulletin and Xbits, you might have stumbled upon me on T3-Design.com where everything used to happen. The last couple of years i was in an on-line hiatus because i was determined to finally take my degree in computer science from the Technological Educational Institute of Athens. This site is my come back to active duty if you will. For this endeavour to succeed i needed to start over from scratch

In the beginning, back in 2005, T3-Design.com was powered by ldu , seditio later and finally wordpress. For years i loathed to have my own base application that i will be able to use for my projects and ideas. For komposta.net i decided to go all out and did that. I started my own CMS or what i prefer to call it CMF,  content management framework. It's called Komposta CMF and as i write this post it's only 96 commits old and hopefully i will make it public later this year.

In the past year i worked a lot with Zend Framework. These were probably the most productive months in my career, so using a php framework was only natural. After research and a lot of consideration i picked Lithium to be the brute force behind komposta for reasons i will explain later...

More info about the Komposta is coming soon for now just have a look

 routes.png resources.png options.png mainSettings.png cron.png adminNavigation.png

I will slowly add all the content from T3-Design for those who miss it, for now just step in and leave a comment please!