Successfully reported this slideshow.
Your SlideShare is downloading. ×

Brighton SEO Sept 2019 PowerShell

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 92 Ad
Advertisement

More Related Content

Slideshows for you (20)

Similar to Brighton SEO Sept 2019 PowerShell (20)

Advertisement

Brighton SEO Sept 2019 PowerShell

  1. 1. @Fearless_Shultz #brightonSEO Utilizing the power of PowerShell for SEO Mike Osolinski // Technical SEO Consultant SLIDESHARE.NET/MikeOsolinski @Fearless_Shultz
  2. 2. @Fearless_Shultz #brightonSEO https://www.slideshare.net/MikeOsolinski/command-line-automation-for-repetitive-tasks
  3. 3. @Fearless_Shultz #brightonSEO What I’m Going to Talk About Today A Brief Introduction to what PowerShell is, it's components and interface Some Examples of Using PowerShell to Automate SEO Tasks Extending PowerShell Beyond It’s Native Capabilities
  4. 4. @Fearless_Shultz #brightonSEO Why Automate SEO Processes?
  5. 5. @Fearless_Shultz #brightonSEO Spend Less Time Gathering and Formatting Data and More Time on Analysis
  6. 6. @Fearless_Shultz #brightonSEO Make your business more profitable by being able to do the same work in a shorter period of time
  7. 7. @Fearless_Shultz #brightonSEO Allow teams to easily handle greater workloads without having to recruit new staff
  8. 8. @Fearless_Shultz #brightonSEO Have an Easier Life…
  9. 9. @Fearless_Shultz #brightonSEO So Why Don’t More People Automate Processes?
  10. 10. @Fearless_Shultz #brightonSEO
  11. 11. @Fearless_Shultz #brightonSEO Lack of Awareness of the potential time savings and that many repetitive tasks can be easily automated
  12. 12. @Fearless_Shultz #brightonSEO Fear either that scripting is too hard for you to learn and you won’t be able to do it or that you might break something *Spoiler – It’s not and You Will. Everyone does
  13. 13. @Fearless_Shultz #brightonSEO Not having / wanting to spend time learning to write scripts
  14. 14. @Fearless_Shultz #brightonSEO Why PowerShell?
  15. 15. @Fearless_Shultz #brightonSEO
  16. 16. @Fearless_Shultz #brightonSEO Already Installed on Most Windows Computers so a lack of complicated set up processes
  17. 17. @Fearless_Shultz #brightonSEO Open Source Meaning it is Cross Platform and has a Large Development Community
  18. 18. @Fearless_Shultz #brightonSEO Makes Working with Data Easy and Reduces the Need to Write Complex Formulas and Expressions
  19. 19. @Fearless_Shultz #brightonSEO Highly Extendable Via Gallery of Custom Modules
  20. 20. @Fearless_Shultz #brightonSEO What is PowerShell?
  21. 21. @Fearless_Shultz #brightonSEO PowerShell Gives You Superpowers!!!
  22. 22. @Fearless_Shultz #brightonSEO Actually, PowerShell is a task-based command-line shell and scripting language built on .NET which helps system administrators and power- users rapidly automate tasks that manage operating systems and processes But that doesn’t sound as cool…
  23. 23. @Fearless_Shultz #brightonSEO So Why Should SEO’s Care?
  24. 24. @Fearless_Shultz #brightonSEO • Scrape Websites • Automate Website Interactions • Run Screaming Frog Reports • Consume Rest APIS • Extract and Manipulate Data • And Much More……..
  25. 25. @Fearless_Shultz #brightonSEO Getting Started
  26. 26. @Fearless_Shultz #brightonSEO https://www.peppercrew.nl/index.php/2014/06/powershell- coding-on-a-mac/ https://wilsonmar.github.io/powershell-on-mac/ https://docs.microsoft.com/en- us/powershell/scripting/install/installing-powershell-core-on- linux?view=powershell-6 https://docs.microsoft.com/en- us/powershell/scripting/install/installing-powershell-core-on- macos?view=powershell-6 https://www.starwindsoftware.com/blog/using-powershell- on-linux
  27. 27. @Fearless_Shultz #brightonSEO Scripting Panel Command Line Cmdlets
  28. 28. @Fearless_Shultz #brightonSEO Cmdlets
  29. 29. @Fearless_Shultz #brightonSEO Command Line A cmdlet (pronounced "command-let") is a lightweight Windows PowerShell script that performs a single function. Invoke-WebRequest Import / Export CSVInvokeRestMethod
  30. 30. @Fearless_Shultz #brightonSEO Command Line https://www.powershellgallery.com
  31. 31. @Fearless_Shultz #brightonSEO Command Line https://www.powershellgallery.com
  32. 32. @Fearless_Shultz #brightonSEO Invoke-WebRequest
  33. 33. @Fearless_Shultz #brightonSEO The Invoke-WebRequest cmdlet sends HTTP and HTTPS requests to a web page or web service. It parses the response and returns collections of links, images, and other significant HTML elements. https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-6
  34. 34. @Fearless_Shultz #brightonSEO $w = Invoke-WebRequest -Uri 'https://moz.com/learn/seo/what-is-seo' $w | Get-Member $w = $w.RawContent $w
  35. 35. @Fearless_Shultz #brightonSEO
  36. 36. @Fearless_Shultz #brightonSEO
  37. 37. @Fearless_Shultz #brightonSEO
  38. 38. @Fearless_Shultz #brightonSEO $w = Invoke-WebRequest -Uri 'https://moz.com/learn/seo/what- is-seo' $w | Get-Member $w = $w.Links $w
  39. 39. @Fearless_Shultz #brightonSEO
  40. 40. @Fearless_Shultz #brightonSEO $w = Invoke-WebRequest -Uri 'https://moz.com/learn/seo/what- is-seo' $w | Get-Member $w = $w.Links | Select innerHTML, href | WHERE innerHTML -like *SEO* $w | Export-Csv 'C:scriptslinks.csv' -NoTypeInformation
  41. 41. @Fearless_Shultz #brightonSEO innerHTML href Free SEO Tools https://moz.com/tools https://moz.com/learn/seo https://moz.com/training https://moz.com/beginners-guide-to-seo SEO Learning Center https://moz.com/learn Free SEO Tools https://moz.com/free-seo-tools What is SEO? https://moz.com/learn/seo/what-is-seo On-Site SEO /learn/seo/on-site Local SEO /learn/seo/local Mobile SEO /learn/seo/mobile International SEO /learn/seo/international Beginner's Guide to SEO /beginners-guide-to-seo
  42. 42. @Fearless_Shultz #brightonSEO $w = Invoke-WebRequest -Uri 'https://moz.com/learn/seo/what- is-seo' $w | Get-Member $w = $w.Links | Select innerHTML $w | Out-File 'C:scriptslink-text.txt' Get-Content C:scriptslink-text.txt | New-WordCloud -Path C:scriptswordcloud.svg -ImageSize 1080p
  43. 43. @Fearless_Shultz #brightonSEO https://github.com/vexx32/PSWordCloud
  44. 44. @Fearless_Shultz #brightonSEO $currentRequest = Invoke-WebRequest -Uri https://moz.com/learn/seo/what-is-seo $currentRequest.AllElements
  45. 45. @Fearless_Shultz #brightonSEO
  46. 46. @Fearless_Shultz #brightonSEO $w = Invoke-WebRequest -Uri 'https://moz.com/learn/seo/what- is-seo' $w.AllElements | where tagname -EQ "p" | select innerText | where innerText -Like *seo*
  47. 47. @Fearless_Shultz #brightonSEO
  48. 48. @Fearless_Shultz #brightonSEO $w = Invoke-WebRequest -Uri 'https://moz.com/learn/seo/what- is-seo' $w.AllElements | where tagname -EQ "p" | select innerText | where innerText -Like *seo*| Out-File 'C:scriptsscriptfilesparagraphs.txt' Get-Content C:scriptsscriptfilesparagraphs.txt | New-WordCloud -Path C:scriptsscriptfilesparagraph-cloud.svg -ImageSize 1080p
  49. 49. @Fearless_Shultz #brightonSEO
  50. 50. @Fearless_Shultz #brightonSEO $path = 'C:scriptsscriptfiles' Import-Csv -Path 'C:scriptsscriptfilesseo-competitors.csv'| ForEach-Object{ foreach ($property in $_.PSObject.Properties) { $currentRequest = Invoke-WebRequest -Uri $property.Value $pageTitle = $currentRequest.ParsedHtml.title $currentSavePath = $path + $pageTitle + '.txt' $currentSavePath = $currentSavePath -replace("?","") $currentSavePath = $currentSavePath -replace("/","") $currentSVGPath = $currentSavePath -replace("txt","svg") $paragraphs = $currentRequest.AllElements | where tagname -EQ "p"| select innerText| Out-File $currentSavePath Get-Content $currentSavePath | New-WordCloud -Path $currentSVGPath -ImageSize 1080p } }
  51. 51. @Fearless_Shultz #brightonSEO
  52. 52. @Fearless_Shultz #brightonSEO Invoke-RestMethod
  53. 53. @Fearless_Shultz #brightonSEO The Invoke-WebRestMethod cmdlet sends HTTP and HTTPS requests to a web page or web service. It parses the response and returns collections of links, images, and other significant HTML elements. https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-6
  54. 54. @Fearless_Shultz #brightonSEO $pageSpeedData = Invoke-RestMethod -Uri 'https://www.googleapis.com/pagespeedonl ine/v5/runPagespeed?url=https://www.brig htonseo.com/'
  55. 55. @Fearless_Shultz #brightonSEO
  56. 56. @Fearless_Shultz #brightonSEO $pageSpeedData = Invoke-RestMethod -Uri 'https://www.googleapis.com/pagespeedonl ine/v5/runPagespeed?url=https://www.brig htonseo.com/' $pageSpeedData.lighthouseResult.audits
  57. 57. @Fearless_Shultz #brightonSEO
  58. 58. @Fearless_Shultz #brightonSEO
  59. 59. @Fearless_Shultz #brightonSEO
  60. 60. @Fearless_Shultz #brightonSEO $pageSpeedData = Invoke-RestMethod -Uri 'https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url =https://www.dailymail.co.uk' $offScreenImageCount = $pageSpeedData.lighthouseResult.audits.'offscreen- images'.details.items.count for($i = 0; $i -le $offScreenImageCount; $i++){ $element = $pageSpeedData.lighthouseresult.audits.'offscreen- images'.details.items.getvalue($i) $element | export-csv 'C:scriptsoffscreen-images.csv' - notypeinformation -append }
  61. 61. @Fearless_Shultz #brightonSEO
  62. 62. @Fearless_Shultz #brightonSEO totalBytes wastedBytes wastedPercent url requestStartTime 43337 43337 100 https://i.dailymail.co.uk/1s/2019/09/04/14/18057384-0-image-m-2_1567604298582.jpg 363649.4834 33206 33206 100 https://i.dailymail.co.uk/1s/2019/09/05/01/18085400-0-image-a-1_1567644632414.jpg 363649.2957 18845 18845 100 https://i.dailymail.co.uk/1s/2019/09/02/16/17978778-0-image-a-36_1567436926465.jpg 363650.0165 18714 18714 100 https://i.dailymail.co.uk/1s/2019/09/03/12/18010078-0-image-a-2_1567509730138.jpg 363650.6832 17963 17963 100 https://i.dailymail.co.uk/1s/2019/09/02/18/17982924-0-image-a-48_1567445797522.jpg 363650.6857 17524 17524 100 https://i.dailymail.co.uk/1s/2019/09/04/09/18050766-0-image-a-12_1567585569391.jpg 363650.6837 14205 14205 100 https://i.dailymail.co.uk/1s/2019/09/04/18/18072588-0-image-a-35_1567619706814.jpg 363650.6849 14101 14101 100 https://i.dailymail.co.uk/1s/2019/09/05/00/18082194-0-image-a-40_1567638380685.jpg 363650.6838 13642 13642 100 https://i.dailymail.co.uk/1s/2019/09/04/14/18062208-0-image-a-30_1567605415945.jpg 363650.6846 13164 13164 100 https://i.dailymail.co.uk/1s/2019/09/02/22/17987956-0-image-a-9_1567458265073.jpg 363650.0176 13063 13063 100 https://i.dailymail.co.uk/i/pix/2018/10/03/tips_v3_opt1_308.png 363649.4832 12987 12987 100 https://i.dailymail.co.uk/1s/2019/09/03/22/18033866-0-image-a-10_1567547692640.jpg 363650.6835
  63. 63. @Fearless_Shultz #brightonSEO Import-Csv 'C:test-urls.csv' | Foreach-Object { foreach ($property in $_.PSObject.Properties) { $currentURL = 'https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=' + $property.Value $testResults = Invoke-restMethod -Uri $currentURL$testResults.lighthouseResult.audits.'render-blocking- resources'.details.items $arrayCount = $testResults.lighthouseResult.audits.'render-blocking- resources'.details.items.Count for($i = 0; $i -le $arrayCount; $i++){$element = $testResults.lighthouseResult.audits.'render-blocking- resources'.details.items.GetValue($i) $element | Export-Csv 'C:scriptsrender-results.csv' -NoTypeInformation - Append } }
  64. 64. @Fearless_Shultz #brightonSEO https://www.programmableweb.com/apis/directory
  65. 65. @Fearless_Shultz #brightonSEO + +
  66. 66. @Fearless_Shultz #brightonSEO # run the screaming frog crawl cd "C:Program Files (x86)Screaming Frog SEO Spider" $env:Path = $env:Path + ";C:Program Files (x86)Screaming Frog SEO Spider" $startingURL = "https://mikeosolinski.co.uk" ScreamingFrogSEOSpiderCli.exe --crawl $startingURL --headless --save-crawl --output-folder “C:scriptssf" --export-tabs "Internal:HTML,Response Codes:Client Error (4xx)" https://www.screamingfrog.co.uk/seo-spider/user-guide/general/#commandlineoptions
  67. 67. @Fearless_Shultz #brightonSEO #create an excel document to contain the data $xl = New-Object -ComObject Excel.Application $xl.Visible = $true #add a workbook to the excel object $workbook1 = $xl.WorkBooks.Add() #add the sheet to contain long page titles $lngPageTitles = $workbook1.Worksheets.Item(1) $lngPageTitles.Name = 'Long Page Titles' #create the column headers for long page titles $lngPageTitles.Cells.Item(1,1) = 'Address' $lngPageTitles.Cells.Item(1,2) = 'Status' $lngPageTitles.Cells.Item(1,3) = 'Title Length'
  68. 68. @Fearless_Shultz #brightonSEO #import the title length data from the screaming frog crawl $titledata = Import-Csv -Path ‘C:scriptssfinternal_html.csv'| select Address, Status,'Title 1 Length'| where Status -eq 'OK'| where 'Title 1 Length' -gt 55 #iterate through the records and add data to the sheet $i = 2 foreach($record in $titledata) { $titledata.Address $titledata.'Title 1 Length' $lngPageTitles.cells.item($i,1) = $record.Address $lngPageTitles.cells.item($i,2) = $record.Status $lngPageTitles.cells.item($i,3) = $record.'Title 1 Length' $i++ }
  69. 69. @Fearless_Shultz #brightonSEO https://github.com/mikeosolinski/powershell/blob/master/screaming-frog-excel
  70. 70. @Fearless_Shultz #brightonSEO https://github.com/mikeosolinski/powershell/blob/master/site-speed-data
  71. 71. @Fearless_Shultz #brightonSEO # CHANGE THE URL BELOW TO THE ONE THAT YOU WANT TO TEST $urltotest = 'https://mikeosolinski.co.uk' #update the path to save files to from C:SiteSpeedDataTOOL-OUTPUT to whatever you want. avoid spaces in path/filenames #any questions on how to run this ping me on twitter at https://twitter.com/Fearless_Shultz https://github.com/mikeosolinski/powershell/blob/master/site-speed-data
  72. 72. @Fearless_Shultz #brightonSEO Extending PowerShell
  73. 73. @Fearless_Shultz #brightonSEO Install-Module -Name GoogleMap -Scope CurrentUser https://geekeefy.wordpress.com/2016/05/17/powershell-module-for-google-map/
  74. 74. @Fearless_Shultz #brightonSEO $env:GoogleGeocode_API_Key = “INSERT API KEY" $env:GooglePlaces_API_Key = “INSERT API KEY" ("Brighton, UK" | Get-GeoCoding).coordinates | Get-NearbyPlace -Radius 1000 -TypeOfPlace hotel
  75. 75. @Fearless_Shultz #brightonSEO
  76. 76. @Fearless_Shultz #brightonSEO Install-Module -Name PSTwitterAPI -Scope CurrentUser https://github.com/mkellerman/PSTwitterAPI
  77. 77. @Fearless_Shultz #brightonSEO Import-Module PSTwitterAPI #set keys $apiKey = INSERT KEY HERE' $apiSecret = INSERT API SECRET HERE' $AccessToken = INSERT ACCESS TOKEN HERE' $tokenSecret = INSERT TOKEN SECRET HERE' # Provide Authentication for the Twitter API # https://twittercommunity.com/t/how-to-get-my-api-key/7033 Set-TwitterOAuthSettings -ApiKey $apiKey -ApiSecret $apiSecret -AccessToken $AccessToken -AccessTokenSecret $tokenSecret # Get user twitter profile $followers = Get-TwitterFollowers_List -count 200 $followers.users.name
  78. 78. @Fearless_Shultz #brightonSEO
  79. 79. @Fearless_Shultz #brightonSEO Import-Module PSTwitterAPI #set keys $apiKey = INSERT KEY HERE' $apiSecret = INSERT API SECRET HERE' $AccessToken = INSERT ACCESS TOKEN HERE' $tokenSecret = INSERT TOKEN SECRET HERE' # Provide Authentication for the Twitter API # https://twittercommunity.com/t/how-to-get-my-api-key/7033 Set-TwitterOAuthSettings -ApiKey $apiKey -ApiSecret $apiSecret -AccessToken $AccessToken -AccessTokenSecret $tokenSecret # Get user twitter profile $followers = Get-TwitterFollowers_List -count 200 $followers.users
  80. 80. @Fearless_Shultz #brightonSEO
  81. 81. @Fearless_Shultz #brightonSEO Import-Module PSTwitterAPI #set keys $apiKey = INSERT KEY HERE' $apiSecret = INSERT API SECRET HERE' $AccessToken = INSERT ACCESS TOKEN HERE' $tokenSecret = INSERT TOKEN SECRET HERE' # Provide Authentication for the Twitter API # https://twittercommunity.com/t/how-to-get-my-api-key/7033 Set-TwitterOAuthSettings -ApiKey $apiKey -ApiSecret $apiSecret -AccessToken $AccessToken -AccessTokenSecret $tokenSecret # Get user twitter profile $followers = Get-TwitterFollowers_List -count 200 $followers.users| select name, screen_name, description| where description -Like '*SEO*' Export-Csv 'C:scriptsseo-followers.csv'-NoTypeInformation
  82. 82. @Fearless_Shultz #brightonSEO
  83. 83. @Fearless_Shultz #brightonSEO
  84. 84. @Fearless_Shultz #brightonSEO Resources
  85. 85. @Fearless_Shultz #brightonSEO https://github.com/mikeosolinski/powershell/ My Github profile which contains commented versions of all of the scripts mentioned https://docs.microsoft.com/en-us/skypeforbusiness/set-up-your-computer-for- windows-powershell/download-and-install-windows-powershell-5-1 The download page to install PowerShell https://www.powershellgallery.com/ PowerShelll Cmdlet Gallery https://www.adamtheautomator.com/invoke-webrequest-powershell/ Guide to Invoke-WebRequest https://www.gngrninja.com/script-ninja/2016/7/8/powershell-getting-started- utilizing-the-web Another guide to invoke WebRequest https://stackoverflow.com/questions/11885246/how-do-i-loop-through-a-line- from-a-csv-file-in-powershell Loops in PowerShell https://www.youtube.com/watch?v=PXBMdIkH24I Associative Arrays in PowerShell https://vwiki.co.uk/MySQL_and_PowerShell Accessing MySQL With PowerShell https://mcpmag.com/articles/2018/08/08/replace-text-with-powershell.aspx Finding and Replacing Text https://techblog.dorogin.com/generate-word-documents-with-powershell- cda654b9cb0e Generating Word Documents with PowerShell https://www.powershellbros.com/powershell-tip-of-the-week-create-invoke- webrequest-from-chrome/ PowerShell and Google Chrome https://docs.google.com/spreadsheets/d/1psz6SvRqv7fjIFIiAySPNshObMINjjFeCvGqTjluGLs/edit?usp=sharing
  86. 86. @Fearless_Shultz #brightonSEO Closing Thoughts
  87. 87. @Fearless_Shultz #brightonSEO PowerShell is Not Just for Systems Administrators and Network Admins
  88. 88. @Fearless_Shultz #brightonSEO PowerShell is An Incredibly Versatile Tool and you are Only Limited By Your Own Imagination
  89. 89. @Fearless_Shultz #brightonSEO Even If You Don’t Want to Learn to Script Yourself you Should Understand the Opportunities
  90. 90. @Fearless_Shultz #brightonSEO But. . .
  91. 91. @Fearless_Shultz #brightonSEO You CAN do it
  92. 92. @Fearless_Shultz #brightonSEO https://mikeosolinski.co.uk http://twitter.com/Fearless_Shultz

×