Magento performance

Magento Enterprise Edition (Magento EE) have a lot of features. One of them is "Full Page Caching" (FPC), which increase store performance and makes store pages to load much faster.

By default Magento EE caching works for category, product, CMS pages. If you want to create your own controller the response will be dynamically generated for it, and FPC won't be used. To add your own controller to FPC you need to add some modifications in your source code.

Lets see how it works. In the Mage_Core_Model_App::run() the following logic is responsible for FPC:

        if ($this->_cache->processRequest()) {
            $this->getResponse()->sendResponse();
        } else {
            ... // page not in the cache processing
        }

In the code above, Magento checks if page is cached and return it without additional resource-heavy initializations:

            // page not in cache processing
            $this->_initModules();
            $this->loadAreaPart(Mage_Core_Model_App_Area::AREA_GLOBAL, Mage_Core_Model_App_Area::PART_EVENTS);

            if ($this->_config->isLocalConfigLoaded()) {
                $scopeCode = isset($params['scope_code']) ? $params['scope_code'] : '';
                $scopeType = isset($params['scope_type']) ? $params['scope_type'] : 'store';
                $this->_initCurrentStore($scopeCode, $scopeType);
                $this->_initRequest();
                Mage_Core_Model_Resource_Setup::applyAllDataUpdates();
            }

            $this->getFrontController()->dispatch();

This is very important part, Magento checks the cache without modules and all other configuration files loading. That is why the FPC increase page loading dramatically. But another side of this is that in your subprocessors/placeholders you should decide whether to use or not to use the cache by request parameters (read further for detailed explanation of this point).

The Mage_Core_Model_App::_cache property is an object of Mage_Core_Model_Cache class, this is global class of the cache. But especially for FPC there is only one method processRequest():

    public function processRequest()
    {
        if (empty($this->_requestProcessors)) {
            return false;
        }

        $content = false;
        foreach ($this->_requestProcessors as $processor) {
            $processor = $this->_getProcessor($processor);
            if ($processor) {
                $content = $processor->extractContent($content);
            }
        }

        if ($content) {
            Mage::app()->getResponse()->appendBody($content);
            return true;
        }
        return false;
    }

As you can see this class just runs the "request processors" classes. For default Magento EE it is only one processor: Enterprise_PageCache_Model_Processor which is set in the enterprise.xml configuration file:

<config>
    <global>
        <cache>
            <request_processors>
                <ee>Enterprise_PageCache_Model_Processor</ee>
            </request_processors>
            ...
        </cache>
        ...
    </global>
</config>

The Enterprise_PageCache_Model_Processor is the main class for FPC in the Magento EE. In the extractContent() method Magento tries to get "subprocesser" class which is responsible for caching controller. When Magento is checking if the cache exists, Magento tries to get this class from metadata:

            $subprocessorClass = $this->getMetadata('cache_subprocessor');

The metadata is saved in the moment when Magento saves page content in the cache, processRequestResponse() method:

            $processor = $this->getRequestProcessor($request);
            if ($processor && $processor->allowCache($request)) {
                $this->setMetadata('cache_subprocessor', get_class($processor));
                ...
            }

In short it works in the following way:

When we load Magento page, in the time between loading of general configuration files (app/etc/*.xml files) and modules data, Magento checks if the requested page is cached. In order to check it Magento uses generated _requestCacheId parameter. This parameter is a hash of the URI path, plus some additional data from cookies - like customer group ID, customer segment ID, currency and etc.

Using the "_requestCacheId" Magento gets metadata (simple (key => value) data). If information for the "_requestCacheId" does not exists Magento exit from cache model and generate the page dynamically. When dynamic generation completed Magento store metadata and page content source code in the cache (if request is used FPC processor).

In next loading of the same page, Magento gets metadata and gets FPC subprocessor. After that subprocessor will return page content if it is possible.

The getRequestProcessor() method is the place where Magento checks if requested path (controller) is used for FPC cache or not:

    public function getRequestProcessor(Zend_Controller_Request_Http $request)
    {
        if ($this->_requestProcessor === null) {
            $this->_requestProcessor = false;
            $configuration = Mage::getConfig()->getNode(self::XML_NODE_ALLOWED_CACHE);
            if ($configuration) {
                $configuration = $configuration->asArray();
            }
            $module = $request->getModuleName();
            $action = $request->getActionName();
            if (strtolower($action) == self::NOT_FOUND_ACTION && isset($configuration['_no_route'])) {
                $model = $configuration['_no_route'];
            } elseif (isset($configuration[$module])) {
                $model = $configuration[$module];
                $controller = $request->getControllerName();
                if (is_array($configuration[$module]) && isset($configuration[$module][$controller])) {
                    $model = $configuration[$module][$controller];
                    if (is_array($configuration[$module][$controller])
                            && isset($configuration[$module][$controller][$action])) {
                        $model = $configuration[$module][$controller][$action];
                    }
                }
            }
            if (isset($model) && is_string($model)) {
                $this->_requestProcessor = Mage::getModel($model);
            }
        }
        return $this->_requestProcessor;
    }

Magento checks "frontend/cache/requests" path in the config xml (XML_NODE_ALLOWED_CACHE constant), for example for categories and CMS pages:

<config>
    <frontend>
        ...
        <cache>
            <requests>
                ...
                <cms>enterprise_pagecache/processor_default</cms>
                <catalog>
                    <category>
                        <view>enterprise_pagecache/processor_category</view>
                    </category>
                </catalog>
                ...
            </requests>
            ...
        </cache>
        ...
    </frontend>
    ...
</config>

You are able to add caching to the whole controller or only to the specific action. If your action show some content which is not depends on parameters (filters, pagination, random, etc) you can use enterprise_pagecache/processor_default FPC processor.

E.g. in our "Magento brands" extension we have added the following setup to the config.xml file:

        <cache>
            <requests>
                <turnkeye_brand>
                    <index>
                        <list>enterprise_pagecache/processor_default</list>
                        <view>turnkeye_brand/pagecache_processor_brand</view>
                    </index>
                </turnkeye_brand>
            </requests>
        </cache>

For brands listing page we use default subprocessor. The reason of this is that we do not have filters sorting or other parameters which change the generated result page.

On the brand page we use layered navigation, and here we need to use our own subprocessor which will store page content depending on parameters of filters/sorting/pagination. You can find good example in the subprocessor class of the category.

When you create your own subprocessor class, extend it from Enterprise_PageCache_Model_Processor_Default. In your subprocessor class you should create two methods: getPageIdInApp() and getPageIdWithoutApp(). The aims of these two methods are self-described. In Magento FPC logic you have page ID as a key and response for it as a value (so this page ID should have parameters with filters/sorting etc). The first method "getPageIdInApp()" should return unique page ID when your page cache is generated (dynamic request). And the second method getPageIdWithoutApp() should generate the same ID in the situation when you check if the cache exists (no modules loaded and only Magento core configuration is initialized).

Our getPageIdInApp() method:

    public function getPageIdInApp(Enterprise_PageCache_Model_Processor $processor)
    {
        $queryParams = $this->_getQueryParams();

        $this->setBrandCookieValue($queryParams);
        $this->_prepareBrandSession();

        $brand = Mage::helper('turnkeye_brand')->getBrand();
        if ($brand) {
            $processor->setMetadata(self::METADATA_BRAND_ID, $brand->getId());
            $this->_updateBrandViewedCookie($processor);
        }

        return $processor->getRequestId() . '_' . md5($queryParams);
    }

First of all we will get all parameters in the requested page, _getQueryParams() method:

    protected function _getQueryParams()
    {
        if (is_null($this->_queryParams)) {
            $queryParams = array_merge($this->_getSessionParams(), $_GET);
            ksort($queryParams);
            $this->_queryParams = json_encode($queryParams);
        }

        return $this->_queryParams;
    }

Here we will get some parameters which stored in the session and apply filter parameters ($_GET variable) to them. After that we will sort array by a key (so if we have different position in filters we will use one cache ID for the same filters values, example "?color=1&size=2" is the same as "?size=2&color=1"). Finally we will return JSON with used parameters on the page. The _getSessionParams() method:

    protected $_paramsMap = array(
        'display_mode'  => 'mode',
        'limit_page'    => 'limit',
        'sort_order'    => 'order',
        'sort_direction'=> 'dir',
    );

    /**
     * Get page view related parameters from session mapped to query parametes
     * @return array
     */
    protected function _getSessionParams()
    {
        $params = array();
        $data   = Mage::getSingleton('turnkeye_brand/session')->getData();
        foreach ($this->_paramsMap as $sessionParam => $queryParam) {
            if (isset($data[$sessionParam])) {
                $params[$queryParam] = $data[$sessionParam];
            }
        }
        return $params;
    }

After getting of all parameters in the getPageIdInApp() method we will save it in the cookie:

    public static function setBrandCookieValue($value)
    {
        setcookie(self::COOKIE_BRAND_PROCESSOR, $value, 0, '/');
    }

The reason of this, is that we need to get session data in the following request, for example if customer move to the next page we should show the page with its own products count, view mode (grid/list), etc.

After that we will save current visibility params ($_paramsMap property) in the session and add information about brand ID to the metadata.

Now lets review the getPageIdWithoutApp():

    public function getPageIdWithoutApp(Enterprise_PageCache_Model_Processor $processor)
    {
        $this->_updateBrandViewedCookie($processor);
        $queryParams = $_GET;

        $sessionParams = $this->getBrandCookieValue();
        if ($sessionParams) {
            $sessionParams = (array)json_decode($sessionParams);
            foreach ($sessionParams as $key => $value) {
                if (in_array($key, $this->_paramsMap) && !isset($queryParams[$key])) {
                    $queryParams[$key] = $value;
                }
            }
        }
        ksort($queryParams);
        $queryParams = json_encode($queryParams);

        $this->setBrandCookieValue($queryParams);

        return $processor->getRequestId() . '_' . md5($queryParams);
    }

We use $_GET as query parameters, after that we get session params, but we are not able to get it from session model, it is not initialized yet, that is why we store this data in the cookie. As the result we have session specified data, plus filters/page, etc parameters.

The last thing we need to specify in the subprocessor is allowCache() method:

    public function allowCache(Zend_Controller_Request_Http $request)
    {
        $res = parent::allowCache($request);
        if ($res) {
            $params = $this->_getSessionParams();
            $queryParams = $request->getQuery();
            $queryParams = array_merge($queryParams, $params);
            $maxDepth = Mage::getStoreConfig(Enterprise_PageCache_Model_Processor::XML_PATH_ALLOWED_DEPTH);
            $res = count($queryParams)<=$maxDepth;
        }
        return $res;
    }

In our case we just check if length of the query params is not more than max allowed value.

If you will complete all changes I described, your controller responses will be cached via Magento EE FPC feature like categories, products, CMS pages. It will increase store performance, in the same way, we do that for our Magento brands extension (we will release this extension in Magento Connect soon).

Also if you use some 3-rd party extensions on your Magento EE store, these modules can disable FPC for some pages. For example, one of very popular extension Category SEO URL do so.

If you wish to use some random data in the pages which uses FPC Magento, in order to allow it (such dynamic blocks can use data from your extensions in online mode) you need to review Magento EE FPC containers logic (Enterprise_PageCache_Model_Container_* classes). We will try to describe it in the next blog articles if it will be interesting.