PrestaShop securityimprovements and optimizations
Group Sitti (www.sitti.fr)
Team of 6 developers & integrators
400 Prestashop installed – ranging from 0.9.6 to 1.3.1
Shared hosting – cluster of 10+ machines (load balancers, web servers, file servers, database servers) About us ?
4 Pillars of performanceInfrastructure(servers, databases)
Our focus: Server-side code (1-st tier, php + sql)
Network, transport protocols
Client-side code (2-nd tier: html + css + javascript) (…) Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: 1premature optimization is the root of all evil.Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.Most important disclaimer (Donald E. Knuth)
Your architecture has to be efficient (good planning)You have to code using best practices (don't do **obviously** stupid things)But prefer rather maintability and readibility of code over the speedWhen speed is not critical (i.e. real time systems, high traffic sites), you can improve it in  later iterationsWhen to optimize?
Measure first! You should know bottlenecks.Benchmark different scenarios and configsGoing Linux? Test Linux, not Win. There are differences Will have 10000 products in your store? Test your modules with db of 10000, not 5Is a 1% improvement worth of additional work?What about 5%? 10%?Try to estimate coding cost vs. hardware costSometimes it's just cheaper to add RAMWhat to optimize?
Small performance gainsUsing (int) instead of intval() can be even 4 X fasterBut overall gain is negligable (unless you are Facebook)Code executed onceTools::setCookieLanguage could be improved, but it is executed onceMythical optimisations ( ” vs ' )But ”$a $b $c” … is faster than $a.” ”.$b.” ”.$cWhatshouldn'tbeoptimised
Server load:ab, siege, multi-mechanize ...Databaseload:MySql Slow Query Log, mysql proxy, ...EXPLAIN PHP:xdebug, dbg, xhprof ...Network / client sideYslow, firebug, WebKitinspector, dynaTrace AJAX, fiddler, google webmaster toolsHow to measure?
Server:Difficult task, often impossible on shared hostingsAsk your adminCPU is rarely a bottleneck, generally indicates problems with suboptimal codeRAM is cheap but not unlimited – attention to memory consuming scriptsTypical problem: gd + jpg -> 2 Mb on disk, 33 Mb decompressed into memoryRamdisk for often accessed, not critical files (frameworks, configuration, tmp) Most common bottleneck: I/O (filesystem, dbs)Improving infrastructure
Every call to fs costs, depending the OS, filesystem and number of files Always use absolute paths in require / includePerformance may start to degrade if you have more than 50 000 files in a directoryEach product has image, each image has 6 thumbnailsDebian + Apache 1.3 (shared hosting, nfs):Filesystem# FilesGlob('*') exec. in sec.file_exists / sec.10004,59360001100013,30210006500055,811475122000142,16718
Directory content splitting:img/p/534-189-small.jpgbecomesimg/p/small/534-189.jpgReading transparently via .htaccessRewriteRule (.*)/p/([^/]*)home\.jpg $1/p/home/$2home.jpgWritingtransparently via class 	if (!imageResize($file, 				$dir.$imageType['name'].'/'.$language['iso_code'].'-default-	'.stripslashes($imageType['name']).'.jpg', ...Solution
Database!Check your indexes!
Avoid to using too many JOINSSELECT * FROM ps_feature` f LEFT JOIN ps_feature_lang` fl ON ( f.`id_feature` = fl.`id_feature` AND fl.`id_lang` = 1) WHERE f.`id_feature` = 1SELECT * FROM ps_feature_lang` fl WHER fl.`id_feature` = 1 AND fl.`id_lang` = 1 VersionTablesColumnsWithout index1.1.0.588458501.2.0.5134670501.3.101356792 (cool! :)
Use VIEWS instead of complicated SELECTSAre you needing ps_connections & ps_connections_page?If you are expecting high traffic, thay can rise 10+ Mb / dayDatabase
Big problem - non unique queries1.3.10, simulation of command process:Index – search – authentication – order (11 pages total) 3001 SQL queries, but only 1314 uniques! (44%) PHP - SQL
Repeatedqueries
Non–optimisedqueries
Best is use mysql proxy or memcachedNot always possibleDo not resolve overhead of unnecessary calls Use internal cacheCan be scoped or globalPrestashop partially uses scoped cacheEasy to implement, tune, and … forgetEach method / class is responsable for caching its query resultsSolutions
static public function getCurrency($id_currency){	return Db::getInstance()->getRow('SELECT * FROM `'._DB_PREFIX_.'currency` 	WHERE `deleted` = 0 AND `id_currency` = '.intval($id_currency));	}static public functiongetCurrency($id_currency){	if (!isset(self::$_cache[$id_currency]))	{		self::$_cache[$id_currency] = Db::getInstance()->getRow('SELECT * FROM `'._DB_PREFIX_.'currency` WHERE `deleted` = 0 AND `id_currency` = '.intval($id_currency));	}	return self::$_cache[$id_currency];	}Scoped cache
Direct in MySqlclass
Catches all output
Harder to implement
Some queries can be repeated but expecting different result (->cart)
Needs kind of "blacklist"
Once implemented, makes application maintenance much easier
Should be implemented as core featureGlobal cache
Regexp is costly, and complicated	return preg_match('/^[a-z0-9!#$%&\'*+\/=?^`{}|~_-]+[.a-z0-	9!#$%&\'*+\/=?^`{}|~_-]*@[a-z0-9]+[._a-z0-9-]*\.[a-z0-9]+$/ui', 	$email);Use filters (> PHP 5.2):	return filter_var($email, FILTER_VALIDATE_EMAIL);Or test before running costly tests:	if (strpos($email, '@')!==false)Use str_replace instead of preg_replace:preg_replace('/"/', '\"', $value)	Faster: str_replace('"', '"', $value)Avoiding regexpSome people, when confronted with a problem, think  “I know, I'll use regular expressions.” Now they have two problems. (jwz)
Avoid capturing groupsreturn preg_match('/^([^<>{}]|<br \/>)*$/ui', $text);	return preg_match('/^(?:[^<>{}]|<br \/>)*$/ui', $text);	?: = non capturing group (no memory allocation!)You often don't need regexpreturn trim($table,'a..zA..Z0..9_') == '';	equals to	return preg_match('/^[a-z0-9_-]+$/ui', $table);	but is up to 2 times faster!Avoidingregexp (2)
foreach($cart->getProducts() as $product)   if ($orderStatus->logable)      ProductSale::addProductSale(intval($product['id_product']), intval($product['cart_quantity']));Should be:if ($orderStatus->logable)     foreach($cart->getProducts() as $product)            ProductSale::addProductSale(intval($product['id_product']), intval($product['cart_quantity']));(no need to test if in every iteration if it does not change)Use conditions wisely
// Send an e-mail to customerif ($id_order_state!= _PS_OS_ERROR_ AND $id_order_state!= _PS_OS_CANCELED_ AND $customer->id){$invoice = new Address(intval($order->id_address_invoice));$delivery = new Address(intval($order->id_address_delivery));$carrier = new Carrier(intval($order->id_carrier));$delivery_state= $delivery->id_state ? new State(intval($delivery->id_state)) : false;$invoice_state= $invoice->id_state ? new State(intval($invoice->id_state)) : false;$data = array( '{firstname}' => $customer->firstname,'{lastname}' => $customer->lastname,'{email}' => $customer->email,'{delivery_company}' => $delivery->company,'{delivery_firstname}' => $delivery->firstname,'{delivery_lastname}' => $delivery->lastname,'{delivery_address1}' => $delivery->address1,'{delivery_address2}' => $delivery->address2,'{delivery_city}' => $delivery->city,'{delivery_postal_code}' => $delivery->postcode,'{delivery_country}' => $delivery->country,'{delivery_state}' => $delivery->id_state ? $delivery_state->name : '','{delivery_phone}' => $delivery->phone,'{delivery_other}' => $delivery->other,'{invoice_company}' => $invoice->company,'{invoice_firstname}' => $invoice->firstname,'{invoice_lastname}' => $invoice->lastname,'{invoice_address2}' => $invoice->address2,'{invoice_address1}' => $invoice->address1,'{invoice_city}' => $invoice->city,'{invoice_postal_code}' => $invoice->postcode,'{invoice_country}' => $invoice->country,'{invoice_state}' => $invoice->id_state ? $invoice_state->name : '','{invoice_phone}' => $invoice->phone,'{invoice_other}' => $invoice->other,{order_name}' => sprintf("#%06d", intval($order->id)),'{date}' => Tools::displayDate(date('Y-m-d H:i:s'), intval($order->id_lang), 1),'{carrier}' => (strval($carrier->name) != '0' ? $carrier->name : Configuration::get('PS_SHOP_NAME')),'{payment}' => $order->payment,Can you spot the problem?
'{products}' => $productsList,'{discounts}' => $discountsList,'{total_paid}' => Tools::displayPrice($order->total_paid, $currency, false, false),'{total_products}' => Tools::displayPrice($order->total_paid - $order->total_shipping - $order->total_wrapping + $order->total_discounts, $currency, false, false),'{total_discounts}' => Tools::displayPrice($order->total_discounts, $currency, false, false),'{total_shipping}' => Tools::displayPrice($order->total_shipping, $currency, false, false),'{total_wrapping}' => Tools::displayPrice($order->total_wrapping, $currency, false, false));if (is_array($extraVars))	$data = array_merge($data, $extraVars);// Join PDF invoiceif (intval(Configuration::get('PS_INVOICE')) AND Validate::isLoadedObject($orderStatus) AND $orderStatus->invoice AND $order->invoice_number){	$fileAttachment['content'] = PDF::invoice($order, 'S');	$fileAttachment['name'] = Configuration::get('PS_INVOICE_PREFIX', intval($order->id_lang)).sprintf('%06d', $order->invoice_number).'.pdf';	$fileAttachment['mime'] = 'application/pdf';}else	$fileAttachment= NULL;if ($orderStatus->send_email AND Validate::isEmail($customer->email))	Mail::Send(intval($order->id_lang), 'order_conf', 'Order confirmation', $data, $customer->email, $customer->firstname.' '.$customer->lastname, NULL, NULL, $fileAttachment);$this->currentOrder = intval($order->id);return true;}$this->currentOrder = intval($order->id);return true;

Good practices for PrestaShop code security and optimization

  • 1.
  • 2.
  • 3.
    Team of 6developers & integrators
  • 4.
    400 Prestashop installed– ranging from 0.9.6 to 1.3.1
  • 5.
    Shared hosting –cluster of 10+ machines (load balancers, web servers, file servers, database servers) About us ?
  • 6.
    4 Pillars ofperformanceInfrastructure(servers, databases)
  • 7.
    Our focus: Server-sidecode (1-st tier, php + sql)
  • 8.
  • 9.
    Client-side code (2-ndtier: html + css + javascript) (…) Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: 1premature optimization is the root of all evil.Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.Most important disclaimer (Donald E. Knuth)
  • 10.
    Your architecture hasto be efficient (good planning)You have to code using best practices (don't do **obviously** stupid things)But prefer rather maintability and readibility of code over the speedWhen speed is not critical (i.e. real time systems, high traffic sites), you can improve it in  later iterationsWhen to optimize?
  • 11.
    Measure first! Youshould know bottlenecks.Benchmark different scenarios and configsGoing Linux? Test Linux, not Win. There are differences Will have 10000 products in your store? Test your modules with db of 10000, not 5Is a 1% improvement worth of additional work?What about 5%? 10%?Try to estimate coding cost vs. hardware costSometimes it's just cheaper to add RAMWhat to optimize?
  • 12.
    Small performance gainsUsing(int) instead of intval() can be even 4 X fasterBut overall gain is negligable (unless you are Facebook)Code executed onceTools::setCookieLanguage could be improved, but it is executed onceMythical optimisations ( ” vs ' )But ”$a $b $c” … is faster than $a.” ”.$b.” ”.$cWhatshouldn'tbeoptimised
  • 13.
    Server load:ab, siege,multi-mechanize ...Databaseload:MySql Slow Query Log, mysql proxy, ...EXPLAIN PHP:xdebug, dbg, xhprof ...Network / client sideYslow, firebug, WebKitinspector, dynaTrace AJAX, fiddler, google webmaster toolsHow to measure?
  • 14.
    Server:Difficult task, oftenimpossible on shared hostingsAsk your adminCPU is rarely a bottleneck, generally indicates problems with suboptimal codeRAM is cheap but not unlimited – attention to memory consuming scriptsTypical problem: gd + jpg -> 2 Mb on disk, 33 Mb decompressed into memoryRamdisk for often accessed, not critical files (frameworks, configuration, tmp) Most common bottleneck: I/O (filesystem, dbs)Improving infrastructure
  • 15.
    Every call tofs costs, depending the OS, filesystem and number of files Always use absolute paths in require / includePerformance may start to degrade if you have more than 50 000 files in a directoryEach product has image, each image has 6 thumbnailsDebian + Apache 1.3 (shared hosting, nfs):Filesystem# FilesGlob('*') exec. in sec.file_exists / sec.10004,59360001100013,30210006500055,811475122000142,16718
  • 16.
    Directory content splitting:img/p/534-189-small.jpgbecomesimg/p/small/534-189.jpgReadingtransparently via .htaccessRewriteRule (.*)/p/([^/]*)home\.jpg $1/p/home/$2home.jpgWritingtransparently via class  if (!imageResize($file, $dir.$imageType['name'].'/'.$language['iso_code'].'-default- '.stripslashes($imageType['name']).'.jpg', ...Solution
  • 17.
  • 18.
    Avoid to usingtoo many JOINSSELECT * FROM ps_feature` f LEFT JOIN ps_feature_lang` fl ON ( f.`id_feature` = fl.`id_feature` AND fl.`id_lang` = 1) WHERE f.`id_feature` = 1SELECT * FROM ps_feature_lang` fl WHER fl.`id_feature` = 1 AND fl.`id_lang` = 1 VersionTablesColumnsWithout index1.1.0.588458501.2.0.5134670501.3.101356792 (cool! :)
  • 19.
    Use VIEWS insteadof complicated SELECTSAre you needing ps_connections & ps_connections_page?If you are expecting high traffic, thay can rise 10+ Mb / dayDatabase
  • 20.
    Big problem -non unique queries1.3.10, simulation of command process:Index – search – authentication – order (11 pages total) 3001 SQL queries, but only 1314 uniques! (44%) PHP - SQL
  • 21.
  • 22.
  • 23.
    Best is usemysql proxy or memcachedNot always possibleDo not resolve overhead of unnecessary calls Use internal cacheCan be scoped or globalPrestashop partially uses scoped cacheEasy to implement, tune, and … forgetEach method / class is responsable for caching its query resultsSolutions
  • 24.
    static public functiongetCurrency($id_currency){ return Db::getInstance()->getRow('SELECT * FROM `'._DB_PREFIX_.'currency` WHERE `deleted` = 0 AND `id_currency` = '.intval($id_currency)); }static public functiongetCurrency($id_currency){ if (!isset(self::$_cache[$id_currency])) { self::$_cache[$id_currency] = Db::getInstance()->getRow('SELECT * FROM `'._DB_PREFIX_.'currency` WHERE `deleted` = 0 AND `id_currency` = '.intval($id_currency)); } return self::$_cache[$id_currency]; }Scoped cache
  • 25.
  • 26.
  • 27.
  • 28.
    Some queries canbe repeated but expecting different result (->cart)
  • 29.
    Needs kind of"blacklist"
  • 30.
    Once implemented, makesapplication maintenance much easier
  • 31.
    Should be implementedas core featureGlobal cache
  • 32.
    Regexp is costly,and complicated return preg_match('/^[a-z0-9!#$%&\'*+\/=?^`{}|~_-]+[.a-z0- 9!#$%&\'*+\/=?^`{}|~_-]*@[a-z0-9]+[._a-z0-9-]*\.[a-z0-9]+$/ui', $email);Use filters (> PHP 5.2): return filter_var($email, FILTER_VALIDATE_EMAIL);Or test before running costly tests: if (strpos($email, '@')!==false)Use str_replace instead of preg_replace:preg_replace('/"/', '\&quot;', $value) Faster: str_replace('"', '&quot;', $value)Avoiding regexpSome people, when confronted with a problem, think  “I know, I'll use regular expressions.” Now they have two problems. (jwz)
  • 33.
    Avoid capturing groupsreturnpreg_match('/^([^<>{}]|<br \/>)*$/ui', $text); return preg_match('/^(?:[^<>{}]|<br \/>)*$/ui', $text); ?: = non capturing group (no memory allocation!)You often don't need regexpreturn trim($table,'a..zA..Z0..9_') == ''; equals to return preg_match('/^[a-z0-9_-]+$/ui', $table); but is up to 2 times faster!Avoidingregexp (2)
  • 34.
    foreach($cart->getProducts() as $product)  if ($orderStatus->logable)      ProductSale::addProductSale(intval($product['id_product']), intval($product['cart_quantity']));Should be:if ($orderStatus->logable)     foreach($cart->getProducts() as $product)            ProductSale::addProductSale(intval($product['id_product']), intval($product['cart_quantity']));(no need to test if in every iteration if it does not change)Use conditions wisely
  • 35.
    // Send ane-mail to customerif ($id_order_state!= _PS_OS_ERROR_ AND $id_order_state!= _PS_OS_CANCELED_ AND $customer->id){$invoice = new Address(intval($order->id_address_invoice));$delivery = new Address(intval($order->id_address_delivery));$carrier = new Carrier(intval($order->id_carrier));$delivery_state= $delivery->id_state ? new State(intval($delivery->id_state)) : false;$invoice_state= $invoice->id_state ? new State(intval($invoice->id_state)) : false;$data = array( '{firstname}' => $customer->firstname,'{lastname}' => $customer->lastname,'{email}' => $customer->email,'{delivery_company}' => $delivery->company,'{delivery_firstname}' => $delivery->firstname,'{delivery_lastname}' => $delivery->lastname,'{delivery_address1}' => $delivery->address1,'{delivery_address2}' => $delivery->address2,'{delivery_city}' => $delivery->city,'{delivery_postal_code}' => $delivery->postcode,'{delivery_country}' => $delivery->country,'{delivery_state}' => $delivery->id_state ? $delivery_state->name : '','{delivery_phone}' => $delivery->phone,'{delivery_other}' => $delivery->other,'{invoice_company}' => $invoice->company,'{invoice_firstname}' => $invoice->firstname,'{invoice_lastname}' => $invoice->lastname,'{invoice_address2}' => $invoice->address2,'{invoice_address1}' => $invoice->address1,'{invoice_city}' => $invoice->city,'{invoice_postal_code}' => $invoice->postcode,'{invoice_country}' => $invoice->country,'{invoice_state}' => $invoice->id_state ? $invoice_state->name : '','{invoice_phone}' => $invoice->phone,'{invoice_other}' => $invoice->other,{order_name}' => sprintf("#%06d", intval($order->id)),'{date}' => Tools::displayDate(date('Y-m-d H:i:s'), intval($order->id_lang), 1),'{carrier}' => (strval($carrier->name) != '0' ? $carrier->name : Configuration::get('PS_SHOP_NAME')),'{payment}' => $order->payment,Can you spot the problem?
  • 36.
    '{products}' => $productsList,'{discounts}'=> $discountsList,'{total_paid}' => Tools::displayPrice($order->total_paid, $currency, false, false),'{total_products}' => Tools::displayPrice($order->total_paid - $order->total_shipping - $order->total_wrapping + $order->total_discounts, $currency, false, false),'{total_discounts}' => Tools::displayPrice($order->total_discounts, $currency, false, false),'{total_shipping}' => Tools::displayPrice($order->total_shipping, $currency, false, false),'{total_wrapping}' => Tools::displayPrice($order->total_wrapping, $currency, false, false));if (is_array($extraVars)) $data = array_merge($data, $extraVars);// Join PDF invoiceif (intval(Configuration::get('PS_INVOICE')) AND Validate::isLoadedObject($orderStatus) AND $orderStatus->invoice AND $order->invoice_number){ $fileAttachment['content'] = PDF::invoice($order, 'S'); $fileAttachment['name'] = Configuration::get('PS_INVOICE_PREFIX', intval($order->id_lang)).sprintf('%06d', $order->invoice_number).'.pdf'; $fileAttachment['mime'] = 'application/pdf';}else $fileAttachment= NULL;if ($orderStatus->send_email AND Validate::isEmail($customer->email)) Mail::Send(intval($order->id_lang), 'order_conf', 'Order confirmation', $data, $customer->email, $customer->firstname.' '.$customer->lastname, NULL, NULL, $fileAttachment);$this->currentOrder = intval($order->id);return true;}$this->currentOrder = intval($order->id);return true;