Just out of the oven – a Zend_Search_Lucene datasource for CakePHP (built with 1.2 but probably works just fine in 1.3) that I originally wrote for an in-house CMS site search plugin. I can’t release the plugin itself (and there’s so much CMS-specific code that it would need a lot of work to make it generic anyway), but I thought that someone might find the datasource itself useful. It’s pretty basic at this point and doesn’t implement some of the fancier Zend_Search_Lucene features such as sorting (it just returns sorted in score order, which is probably what you want anyway).

Zend_Search_Lucene is a text-based search index system for developers who don’t want to (or can’t) use a database for search indexing.

Download the current version of the ZendSearchLuceneDatsource from my Github repository.

I won’t go into detail about how to add data into the Lucene database since the Zend Framework documention is so good (CakePHP should be jealous!). You’ll find all the info you need there. There are also a couple of older articles out there that show how you can integrate Zend_Search_Lucene into CakePHP:

• Build your own search engine in PHP
• Integrating Zend Framework Lucene with your Cake Application

Setup

First, copy zend_search_lucene.php to models/datasources.

Then, you’ll need to download the Zend_Search_Lucene library from the Zend Framework website and put some files into your /vendors directory:

• Zend/Search (the directory and all of its contents)
• Zend/Exception.php

You’ll also need to update your include path to include app/vendors, since the Zend Framework loads a lot of classes on its own. I also made a little autoload function to make the loading of Zend Framework classes easier. Put the following code somewhere common, such as app/bootstrap.php:

ini_set('include_path', ini_get('include_path') . ':' . CAKE_CORE_INCLUDE_PATH . DS . '/vendors');
function __autoload($path) {
if (substr($path, 0, 5) == 'Zend_') {
include str_replace('_', '/', $path) . '.php';
}
return $path;
}

You also need to put the DB config for the datasource in config/database.php (updated Jan 20/2010 for better DebugKit compatibility):

var $zendSearchLucene = array(
	'datasource' => 'ZendSearchLucene',
	'indexFile' => 'lucene', // stored in the cache dir.
	'driver' => '',
	'source' => 'search_indices'
);

Then, in the model that’ll act as your search index (say, for example, SearchIndex), specify the DB config:

<?php class SearchIndex extends AppModel {
var $useDbConfig = 'zendSearchLucene';
}
?>

Saving/Indexing

I’ve tried to keep the datasource functions as simple and familiar as possible. When saving an item to the index, the datasource expects a multidimensional array for each item. For compatibility with CakePHP’s datasource code, the ‘meat’ of the data is nested in the third level of the array. Each sub-array contains information about a field to be stored. For example:

$saveData = array('SearchIndex' => array(
  	'document' => array(
		array(
			'key' => 'name',
			'value' => $record[$Model->alias][$this->settings[$Model->alias]['name']],
			'type' => 'Text'
		),
		array(
			'key' => 'description',
			'value' => $record[$Model->alias][$this->settings[$Model->alias]['description']],
			'type' => 'Text'
		),
		array(
			'key' => 'url',
			'value' => $this->__constructUrl($Model, $record),
			'type' => 'Text'
		)
	)
 ));

Passing that data in a Model::save() call will in turn execute the following Zend code (more or less – this is a very simplified version of the actual ZendSearchLuceneSource saving code):

$index = Zend_Search_Lucene::open('/path/to/the/index/set/in/dbConfig');
$doc = new Zend_Search_Lucene_Document();
foreach ($data as $field) {
$doc->addField(Zend_Search_Lucene_Field::$field['type']($field['key'], $field['value']));
}
$index->addDocument($doc);

Obviously that’s a basic example; you’ll probably send a whole bunch of dynamic info to the indexer. But that’s the gist of it anyway.

Querying

You can search for records just like you would a regular datasource. Pass the search terms as a “query” condition. If you want the search terms to be highlighted in the returned results, pass ‘highlight’ => true in the array of options. Note that only indexed fields will be highlighted.

You can find all results:

function search($term) {
$results = $this->SearchIndex->find('all', array('highlight' => true, 'conditions' => array('query' => 'best cakephp tutorials')));
}

You can mimic Google’s I’m Feeling Lucky with find(‘first’):

function search($term) {
$topResult = $this->SearchIndex->find('first', array('conditions' => array('query' => 'best cakephp tutorials')));
}

You can even paginate:

function search($term) {
$this->paginate = array(
'limit' => 10,
'conditions' => array('query' => 'best CakePHP tutorials'),
'highlight' => true
);

$results = $this->paginate();
}

Results are returned in the expected CakePHP way, as a multidimensional array – $results[0]['MyModelAlias'] for multiple records, $results['MyModelAlias'] for one (i.e. with find(‘first’)).

There you go – enjoy! As always, comments and suggestions are welcomed.

I used the RSS Feed datasource by Loadsys as a guide to good datasource design. I may have borrowed a function or two.

Neil Crookes’ Searchable plugin also helped.

Router::connect('/:form/', array('plugin' => 'email_forms', 'controller' => 'email_forms', 'action' => 'view'), array('form' => '[a-z0-9_-]+'));

This didn’t work, however; the route would take me to the correct action, but the ‘form’ param would always be empty – or so I thought. After doing some checking, mostly thanks to the Debug Kit plugin, I realized that the ‘form’ param is passed with every request. So, I just renamed ‘form’ to ‘emailForm’ and it worked like a charm:

Router::connect( '/:emailForm/', array('plugin' => 'email_forms', 'controller' => 'email_forms', 'action' => 'view'), array('emailForm' => '[a-z0-9_-]+'));

So, watch out for those predefined variables.

The simplest fix is the requestAction method, but that results in a lot of overhead since it results in an almost-complete page load with every use (basically, it just loads another URL on the fly). I searched around the popular CakePHP blogs for a few hours looking for a good solution, but, after not finding anything I liked, I decided to make my own. I actually got the idea for this component from the comments on debuggable.com‘s post ‘requestAction considered harmful‘ -Rafael Bandeira suggested a Widget view helper, which I thought was a great idea but decided to go with a controller component instead.

And that’s where the Widget component comes in. Basically, this component, which I attach to AppController, allows you to retrieve any information from any model, without directly interacting with the model itself and without using $uses, App::import, ClassRegistry, etc. The basic idea is that you write your own findX methods in your models, which you then call via the Widget component.

So, we’re going to work through implementing a Widget component, and we’ll use it to get the latest three published entries from the Articles table and display them across the site. Here’s what we’ll do:

1. Implement support for custom find methods (skip this step if you’ve already done this)
2. Write a method to find the latest Articles
3. Write our Widget Component
4. Add the Widget Component to AppController and write a function to set site-wide dynamic data

1. Implement support for custom find methods

Most CakePHP developers who’ve graduated from the ‘novice’ school have done this, but for those who haven’t: adding custom find methods is a great way to extend your application. And, it’s easy! I’m using the method found in Matt Curry‘s excellent (and free!) Super Awesome Advanced CakePHP Tips e-book.

We just need to overwrite the default Model::find() method with a new find() in our AppModel. It looks something like this:

// For custom find('xxx') methods. The function looks for a __findFindType class method.
function find($type, $options = array()) {
     $method = null;
     if (is_string($type)) {
         $method = sprintf('__find%s', Inflector::camelize($type));
     }

     if ($method && method_exists($this, $method)) {
         return $this->{$method}($options);
     } else {
        $args = func_get_args();
        return call_user_func_array(array('parent', 'find'), $args);
     }
}

Basically, whenever we call find(), we’re going to look in the model doing the finding for a __findXXX method that matches find(‘XXX’). If the method doesn’t exist then we fall back on the default find() method.

2. Write a sample custom find method

Now we’ll write a method in the Article model – /app/models/article.php – class to find the latest articles:

function __findLatest($limit = 3) {
    return $this->find('all', array('limit' => 3, 'published' => true, 'order' => 'Article.begin_publishing DESC'));
}

Pretty simple. To use it, we just call: $this->Article->find(‘latest’);

Write our Widget component

Finally, the good stuff! The Widget component is simple too; it only has one method (of course, expand as you see fit). Put this in /app/controllers/components/widget.php:

<?php
class WidgetComponent extends Object {

    function retrieve($modelName, $findType, $options = NULL) {
        $model = ClassRegistry::init($modelName);
        return $model->find($findType, $options);
    }

}
?>

As you can see, we’re using a method called ‘retrieve’ to find our data. Its first argument is the name of the model from which you want to grab data, while the second argument denotes the name of the findXXX method. The third argument is any optional data you want to pass along. You’ll see an example usage in the next section. Speaking of which…

Add the Widget Component to AppController and write a function to set site-wide dynamic data

So now that we’ve written our custom find method and Widget Component, the only thing left to do is add the functionality to AppController. First, add it to the $components array in /app/app_controller.php:

var $components = array('Widget');

Next, write a function in AppController to set your common (site-wide) widgets. I like to call it “_setCommonWidgets()”:

function _setCommonWidgets()
{
    $this->set('latestArticles', $this->Widget->retrieve('Article', 'latest'));
}

And there’s the Widget Component in action. We call the retrieve() method, pass along the model name we want – Article – and the find type, ‘latest’. We set the resulting array as view variable, so it’ll be available in any view as $latestArticles. But wait!

We forgot one step: we need to invoke the _setCommonWidgets() function. Easy stuff: just call it in AppController::beforeRender():

// Set any common 'widget' variables.
$this->_setCommonWidgets();

And that’s it. Now you can use _setCommonWidgets to give your views access to any data you’d like, without resorting to requestAction, $uses, manual calls to App::import or ClassRegistry, etc.

I’d love feedback on this feature. Improvements especially are welcome.

public static function autoload($path)
{
    include str_replace('_', '/', $path) . '.php';
    return $path;
}

Then when you’re running the bootstrap, change your default registerAutoload() call to:

require_once 'Zend/Loader.php';
Zend_Loader::registerAutoload('Bootstrap');

And that’s it! Erase or comment out those annoying and inefficient require_once statementse. I know, slightly ironic that we need a require_once since the goal is to get rid of them. I never said it was perfect – almost.