Page History

What is Solr Search Engine Bundle?

ezplatform-solr-search-engine

,as the package is called, aims to be a transparent drop-in replacement for the SQL-based "Legacy" search engine powering eZ Platform Search API by default.

When you enable Solr and re-

index your content, all your

existing Search queries using SearchService

will be powered by Solr automatically. This allows you to scale up your eZ Platform installation and be able to continue development locally against SQL engine, and have a test infrastructure, Staging and Prod powered by Solr.

This removes considerable load from your database

. See Architecture for further information on the architecture of eZ Platform.

Status of features:

•  Able to handle all eZ Platform queries
  •  Much more suitable for handling field criteria (performance)
  •  Scoring for content queries and sorting by them by default
•  Indexing plugins (Solr Bundle >= v1.2)
•  Solr 6 support (Solr Bundle >= v1.3)
  •  Scoring for Location queries and sorting by them by default

• Work in progress:
  •  Faceting (possible to write your own since v1.0, ContentType/Section/User implemented since v1.4, suggested further changes to the API for Faceting can be found here)
  •  Index time Boosting (Solr Bundle >= v1.4)
• Future:
  •  Solr cloud support
  •  Highlighting
  •  Spell checking
    
  •  Query time Boosting

How to set up Solr Search engine

Step

0:

Enable Solr Bundle

1. Check in composer.json if you have the ezsystems/ezplatform-solr-search-engine package, if not add/update composer dependencies:

   Code Block
   language bash title command line
   composer require --no-update ezsystems/ezplatform-solr-search-engine:~1.0 composer update

1. Make sure EzPublishSolrSearchEngineBundle is activated with the following line in the app/AppKernel.php file: new EzSystems\EzPlatformSolrSearchEngineBundle\EzSystemsEzPlatformSolrSearchEngineBundle()

Step

1: Configuring & Starting Solr

Download and configure

Solr 4.10.4

First, download and extract Solr

. Solr Bundle 1.x supports Solr 4.10.4:

• solr-4.10.4.tgz or solr-4.10.4.zip

Secondly, copy configuration files needed for eZ Solr Search Engine bundle,

in the example below from the root of your project to the place you extracted Solr:

# Make sure to 

replace the /opt/solr/ path with where you have placed Solr
cp -R vendor/ezsystems/ezplatform-solr-search-engine/lib/Resources/config/solr/* /opt/solr/example/solr/collection1/conf/

/opt/solr/bin/solr start -f 

Solr 6

• solr-6.4.2.tgz or solr-6.4.2.zip

Secondly, copy configuration files needed for eZ Solr Search Engine bundle, here from the root of your project to the place you extracted Solr:

# Make sure to replace the /opt/solr/ path with where you have placed Solr
mkdir -p /opt/solr/server/ez/template
cp -R vendor/ezsystems/ezplatform-solr-search-engine/lib/Resources/config/solr/* /opt/solr/server/ez/template
cp /opt/solr/server/solr/configsets/basic_configs/conf/{currency.xml,solrconfig.xml,stopwords.txt,synonyms.txt,elevate.xml} /opt/solr/server/ez/template
cp /opt/solr/server/solr/solr.xml /opt/solr/server/ez

# Modify solrconfig.xml to remove the section that doesn't agree with your schema
sed -i.bak '/<updateRequestProcessorChain name="add-unknown-fields-to-the-schema">/,/<\/updateRequestProcessorChain>/d' /opt/solr/server/ez/template/solrconfig.xml
 
# Start Solr (but apply autocommit settings below first if you need to)
/opt/solr/bin/solr -s ez
/opt/solr/bin/solr create_core -c collection1 -d server/ez/template

Further configuration

Thirdly, on both Solr 4 and 6 Solr the bundle does not commit solr index changes directly on repository updates, leaving it up to you to tune this using solrconfig.xml as best practice suggests, for example:

<autoCommit>
  <!-- autoCommit is here left as-is like it is out of the box in Solr, this controls hard commits for durability/replication -->
  <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> 
  <openSearcher>false</openSearcher> 
</autoCommit>

<autoSoftCommit>
  <!-- Soft commits controls mainly when changes becomes visible, by default we change value from -1 (disabled) to 100ms, to try to strike a balance between Solr performance and staleness of HttpCache generated by Solr queries -->
  <maxTime>${solr.autoSoftCommit.maxTime:100}</maxTime> 
</autoSoftCommit>

Step 2: Configuring bundle

The Solr search engine bundle can be configured in many ways

. The config further below assumes you have parameters set up for solr dsn and search engine (however both are optional), for example:

    search_engine: solr
    solr_dsn: 'http://localhost:8983/solr'

On to configuring the bundle.

Single Core example (default)

Out of the box in eZ Platform the following is enabled for a simple setup:

ez_search_engine_solr:
    endpoints:
        endpoint0:
            dsn: %solr_dsn%
            core: collection1
    connections:
        default:
            entry_endpoints:
                - endpoint0
            mapping:
                default: endpoint0

The following is an example of configuring Solr Search Engine, where connection name is same as in the example above, and engine is set to solr:

Step 4: Clear prod cache

While Symfony dev environment keeps track of changes to yml files, prod does not, so to make sure Symfony reads the new config we clear cache:

php app/console --env=prod cache:clear

Step 5: Run CLI indexing command

The last step is to execute the initial indexation of data:

php app/console --env=prod --siteaccess=<name> ezplatform:solr_create_index

php app/console --env=prod --siteaccess=<name> ezplatform:reindex

Configuring the Solr Search engine Bundle

Boost configuration

Boosting tells the search engine which parts of the content model have more importance when searching, and is an important part of tuning your search results relevance. Importance is defined using a numeric value, where 1.0 is default, values higher than that are more important, and values lower (down to 0.0) are less important.

Boosting is configured per connection that you configure to use for a given repository, like in the example below:

ez_search_engine_solr:
    connections:
        default:
            boost_factors:
                content_type:
                    # Boost a whole Content Type
                    article: 2.0
                field_definition:
					# Boost a content Field system wide, or for a given Content Type
                    title: 3.0
                    blog_post:
                        # Don't boost title of blog posts that high, but still higher than default
                        title: 1.5
                meta_field:
					# Boost a meta Field (name, text) system wide, or for a given Content Type
                    name: 10.0
                    article:
                        # Boost the meta full text Field for article more than 2.0 set above
                        text: 5.0

• article/title: 2.0
• news/title: 3.0
• blog_post/title:  1.5
• news/description:  1.0 (default)
• article/text (meta): 5.0
• blog_post/name  (meta):  10.0
• article/name  (meta):  2.0

Extending the Solr Search engine Bundle

Document Field Mappers

• all block documents (meaning Content and its Locations, all translations)
• all block documents per translation
• Content documents
• Content documents per translation
• Location documents

Indexing additional data is done by implementing a document field mapper and registering it at one of the five extension points described above. You can create the field mapper class anywhere inside your bundle, as long as when you register it as a service, the "class" parameter" in your services.yml matches the correct path. We have three different field mappers. Each mapper implements two methods, by the same name, but accepting different arguments:

• ContentFieldMapper
  • ::accept(Content $content)
  • ::mapFields(Content $content)
• ContentTranslationFieldMapper
  • ::accept(Content $content, $languageCode)
  • ::mapFields(Content $content, $languageCode)
• LocationFieldMapper
  • ::accept(Location $content)
  • ::mapFields(Location $content)

These can be used on the extension points by registering them with the container using service tags, as follows:

• all block documents
  • ContentFieldMapper
  • ezpublish.search.solr.document_field_mapper.block
• all block documents per translation
  • ContentTranslationFieldMapper
  • ezpublish.search.solr.field_mapper.block_translation
• Content documents
  • ContentFieldMapper
  • ezpublish.search.solr.document_field_mapper.content
• Content documents per translation
  • ContentTranslationFieldMapper
  • ezpublish.search.solr.field_mapper.content_translation
• Location documents
  • LocationFieldMapper
  • ezpublish.search.solr.field_mapper.location

The following example shows how to index data from the parent Location content, in order to make it available for full text search on the children content. A concrete use case could be indexing webinar data on the webinar events, which are children of the webinar. Field mapper could then look something like this:

 <?php

namespace My\WebinarApp;

use EzSystems\EzPlatformSolrSearchEngine\FieldMapper\ContentFieldMapper;
use eZ\Publish\SPI\Persistence\Content\Handler as ContentHandler;
use eZ\Publish\SPI\Persistence\Content\Location\Handler as LocationHandler;
use eZ\Publish\SPI\Persistence\Content;
use eZ\Publish\SPI\Search;

class WebinarEventTitleFulltextFieldMapper extends ContentFieldMapper
{
    /**
     * @var \eZ\Publish\SPI\Persistence\Content\Type\Handler
     */
    protected $contentHandler;

    /**
     * @var \eZ\Publish\SPI\Persistence\Content\Location\Handler
     */
    protected $locationHandler;

    /**
     * @param \eZ\Publish\SPI\Persistence\Content\Handler $contentHandler
     * @param \eZ\Publish\SPI\Persistence\Content\Location\Handler $locationHandler
     */
    public function __construct(
        ContentHandler $contentHandler,
        LocationHandler $locationHandler
    ) {
        $this->contentHandler = $contentHandler;
        $this->locationHandler = $locationHandler;
    }

    public function accept(Content $content)
    {
        // ContentType with ID 42 is webinar event
        return $content->versionInfo->contentInfo->contentTypeId == 42;
    }

    public function mapFields(Content $content)
    {
        $mainLocationId = $content->versionInfo->contentInfo->mainLocationId;
        $location = $this->locationHandler->load($mainLocationId);
        $parentLocation = $this->locationHandler->load($location->parentId);
        $parentContentInfo = $this->contentHandler->loadContentInfo($parentLocation->contentId);

        return [
            new Search\Field(
                'fulltext',
                $parentContentInfo->name,
                new Search\FieldType\FullTextField()
            ),
        ];
    }
}

Since we index full text data only on the Content document, you would register the service like this:

my_webinar_app.webinar_event_title_fulltext_field_mapper:
    class: My\WebinarApp\WebinarEventTitleFulltextFieldMapper
    arguments:
        - '@ezpublish.spi.persistence.content_handler'
        - '@ezpublish.spi.persistence.location_handler'
    tags:
        - {name: ezpublish.search.solr.field_mapper.content}

Providing feedback

After completing the installation you are now free to use your site as usual. If you get any exceptions for missing features, have feedback on performance, or want to discuss, join our community slack channel at https://ezcommunity.slack.com/messages/ezplatform-use/