The document describes the process for summarizing articles using a readability algorithm. It involves: 1. Removing JavaScript and CSS, and filtering out unnecessary nodes based on regex rules. 2. Calculating a score for each node based on tag names, IDs, classes and other attributes. 3. Adjusting scores based on things like number of commas, text length, and transferring scores to parent nodes. 4. Adjusting final candidate scores based on link density, where good content should have less than 5% of text in links. 5. Selecting the top candidate and related sibling nodes as the summary.

1. Readability

2. 規則

3. 訂定關 字
• unlikelyCandidates
• okMaybeItsACandidate
• positive
• negative

4. 訂定關 字
1. public $regexps = array(
2. 'unlikelyCandidates' => '/combx|comment|community|disqus|extra|foot|header|menu|
remark|rss|shoutbox|sidebar|sponsor|ad-break|agegate|pagination|pager|popup|tweet|
twitter/i',
3.
4. 'okMaybeItsACandidate' => '/and|article|body|column|main|shadow/i',
5. 'positive' => '/article|body|content|entry|hentry|main|page|pagination|post|text|blog|
story/i',
6. ...
7. ...
8. );

5. 準備步驟
• 去除 Javascript 與 css
• 根據關 字清除不需要的 node
• 計算 node 得分

6. Node 分數計算

7. Tag name 調整權重
1. switch (strtoupper($node->tagName)) {
2. case 'DIV':
3. $readability->value += 5;
4. break;
5.
6. case 'PRE':
7. case 'TD':
8. case 'BLOCKQUOTE':
9. $readability->value += 3;
10. break;
11.
12. case 'ADDRESS':
13. case 'OL':
14. case 'UL':
15. case 'DL':
16. case 'DD':
17. case 'DT':
18. case 'LI':
19. case 'FORM':
20. $readability->value -= 3;
21. break;
22.
23. case 'H1':
24. case 'H2':
25. case 'H3':
26. case 'H4':
27. case 'H5':
28. case 'H6':
29. case 'TH':
30. $readability->value -= 5;
31. break;
32. }

8. id 與 class 關 字權重
1. /* Look for a special classname */
2. if ($e->hasAttribute('class') && $e->getAttribute('class') != '')
3. {
4. if (preg_match($this->regexps['negative'], $e->getAttribute('class'))) {
5. $weight -= 25;
6. }
7. if (preg_match($this->regexps['positive'], $e->getAttribute('class'))) {
8. $weight += 25;
9. }
10. }

9. 權重調整
• 號分隔數
• 內文長度
• 分數向父元素轉移

10. 1. $contentScore = 0;
2.
3. /* Add a point for the paragraph itself as a base. */
4. $contentScore++;
5.
6. /* Add points for any commas within this paragraph */
7. $contentScore += count(explode(',', $innerText));
8.
9. /* For every 100 characters in this paragraph, add another point. Up to 3 points. */
10.$contentScore += min(floor(strlen($innerText) / 100), 3);
11.
12./* Add the score to the parent. The grandparent gets half. */
13.$parentNode->getAttributeNode('readability')->value += $contentScore;
14.
15.if ($grandParentNode) {
16. $grandParentNode->getAttributeNode('readability')->value += $contentScore/2;
17.}

11. 外部連接權重
• 好的內容有一定比例的連結
• 5% 或更少

12. 外連比例分數調整
1. /**
2. * Scale the final candidates score based on link density. Good content should have a
3. * relatively small link density (5% or less) and be mostly unaffected by this operation.
4. **/
5. $readability = $candidates[$c]->getAttributeNode('readability');
6. $readability->value = $readability->value * (1-$this->getLinkDensity($candidates[$c]));
7.

13. 外連比例分數調整
1. /**
2. * Get the density of links as a percentage of the content
3. * This is the amount of text that is inside a link divided by the total text in the node.
4. *
5. * @param DOMElement $e
6. * @return number (float)
7. */
8. public function getLinkDensity($e) {
9. $links = $e->getElementsByTagName('a');
10. $textLength = strlen($this->getInnerText($e));
11. $linkLength = 0;
12. for ($i=0, $il=$links->length; $i < $il; $i++)
13. {
14. $linkLength += strlen($this->getInnerText($links->item($i)));
15. }
16. if ($textLength > 0) {
17. return $linkLength / $textLength;
18. } else {
19. return 0;
20. }
21. }

14. 最後的最後
• 我們將取得所有候選區塊的權重
• 找出我們的 Top Candidate
• 找出 SiblingNode 運用權重決定關連性
• Clear!

15. Thanks!
• http://arc90.com/
• http://code.google.com/p/arc90labs-readability/
• http://www.readability.com
• http://www.keyvan.net/2010/08/php-readability/