![\"\"\/](\"https:\/\/pbs.twimg.com\/profile_images\/1097426990699855873\/lEI3EWIL_normal.png\")
<\/a><\/figure>\n\n\n\n John <\/strong>\u2714<\/strong>@JohnMu<\/a><\/a><\/p>\n\n\n\n The web has moved from plain HTML – as an SEO you can embrace that. Learn from JS devs & share SEO knowledge with them. JS’s not going away.247<\/a>3:41 PM – Aug 8, 2017<\/a>Twitter Ads info and privacy<\/a>192 people are talking about this<\/a><\/p>\n\n\n\n I\u2019m not saying that SEOs need to go out and learn how to program JavaScript. It\u2019s quite the opposite. SEOs mostly need to know how Google handles JavaScript and how to troubleshoot issues. In very few cases will an SEO even be allowed to touch the code. My goal with this post is to help you learn:<\/p>\n\n\n\n <\/a><\/p>\n\n\n\n JavaScript SEO is a part of Technical SEO<\/a> (Search Engine Optimization) that seeks to make JavaScript-heavy websites easy to crawl and index, as well as search-friendly. The goal is to have these websites be found and rank higher in search engines<\/a>.<\/p>\n\n\n\n Is JavaScript bad for SEO; is JavaScript evil? Not at all. It\u2019s just different from what many SEOs are used to, and there\u2019s a bit of a learning curve. People do tend to overuse it for things where there\u2019s probably a better solution, but you have to work with what you have at times. Just know that Javascript isn\u2019t perfect and it isn\u2019t always the right tool for the job. It can\u2019t be parsed progressively, unlike HTML and CSS, and it can be heavy on page load and performance. In many cases, you may be trading performance for functionality.<\/a><\/p>\n\n\n\n In the early days of search engines, a downloaded HTML response was enough to see the content of most pages. Thanks to the rise of JavaScript, search engines now need to render many pages as a browser would so they can see content how a user sees it.<\/p>\n\n\n\n The system that handles the rendering process at Google is called the Web Rendering Service (WRS). Google has provided a simplistic diagram to cover how this process works.<\/p>\n\n\n\n Source<\/a><\/p>\n\n\n\n Let\u2019s say we start the process at URL.<\/p>\n\n\n\n The crawler sends GET requests to the server. The server responds with headers and the contents of the file, which then gets saved.<\/p>\n\n\n\n The request is likely to come from a mobile user-agent since Google is mostly on mobile-first indexing<\/a> now. You can check to see how Google is crawling your site with the URL Inspection Tool<\/a> inside Search Console<\/a>. When you run this for a URL, check the Coverage information for \u201cCrawled as,\u201d and it should tell you whether you\u2019re still on desktop indexing or mobile-first indexing.<\/p>\n\n\n\n The requests mostly come from Mountain View, CA, USA, but they also do some crawling for locale-adaptive pages<\/a> outside of the United States. I mention this because some sites will block or treat visitors from a specific country or using a particular IP in different ways, which could cause your content not to be seen by Googlebot.<\/p>\n\n\n\n Some sites may also use user-agent detection to show content to a specific crawler. Especially with JavaScript sites, Google may be seeing something different than a user. This is why Google tools such as the URL Inspection Tool inside Google Search Console, the Mobile-Friendly Test<\/a>, and the Rich Results Test<\/a> are important for troubleshooting JavaScript SEO issues. They show you what Google sees and are useful for checking if Google may be blocked and if they can see the content on the page. I\u2019ll cover how to test this in the section about the Renderer because there are some key differences between the downloaded GET request, the rendered page, and even the testing tools.<\/p>\n\n\n\n It\u2019s also important to note that while Google states the output of the crawling process as \u201cHTML\u201d on the image above, in reality, they\u2019re crawling and storing all resources needed to build the page. HTML pages, Javascript files, CSS, XHR requests, API endpoints, and more.<\/p>\n\n\n\n There are a lot of systems obfuscated by the term \u201cProcessing\u201d in the image. I\u2019m going to cover a few of these that are relevant to JavaScript.<\/p>\n\n\n\n Google does not navigate from page to page as a user would. Part of Processing is to check the page for links to other pages and files needed to build the page. These links are pulled out and added to the crawl queue, which is what Google is using to prioritize and schedule crawling.<\/p>\n\n\n\n Google will pull resource links (CSS, JS, etc.) needed to build a page from things like Good:<\/strong><\/p>\n\n\n\n Bad:<\/strong><\/p>\n\n\n\n It\u2019s also worth noting that internal links<\/a> added with JavaScript will not get picked up until after rendering. That should be relatively quick and not a cause for concern in most cases.<\/p>\n\n\n\n Every file that Google downloads, including HTML pages, JavaScript files, CSS files, etc., is going to be aggressively cached. Google will ignore your cache timings and fetch a new copy when they want to. I\u2019ll talk a bit more about this and why it\u2019s important in the Renderer section.<\/p>\n\n\n\n Duplicate content<\/a> may be eliminated or deprioritized from the downloaded HTML before it gets sent to rendering. With app shell models, very little content and code may be shown in the HTML response. In fact, every page on the site may display the same code, and this could be the same code shown on multiple websites. This can sometimes cause pages to be treated as duplicates and not immediately go to rendering. Even worse, the wrong page or even the wrong site may show in search results. This should resolve itself over time but can be problematic, especially with newer websites.<\/p>\n\n\n\n Google will choose the most restrictive statements<\/a> between HTML and the rendered version of a page. If JavaScript changes a statement and that conflicts with the statement from HTML, Google will simply obey whichever is the most restrictive. Noindex will override index, and noindex in HTML will skip rendering altogether.<\/p>\n\n\n\n Every page goes to the renderer now. One of the biggest concerns from many SEOs with JavaScript and two-stage indexing (HTML then rendered page) is that pages might not get rendered for days or even weeks. When Google looked into this, they found pages went to the renderer at a median time of 5 seconds<\/a>, and the 90th percentile was minutes. So the amount of time between getting the HTML and rendering the pages should not be a concern in most cases.<\/p>\n\n\n\n The renderer is where Google renders a page to see what a user sees. This is where they\u2019re going to process the JavaScript and any changes made by JavaScript to the Document Object Model (DOM)<\/a>.<\/p>\n\n\n\n For this, Google is using a headless Chrome browser that is now \u201cevergreen,\u201d which means it should use the latest Chrome version and support the latest features. Until recently, Google was rendering with Chrome 41, so many features were not supported.<\/p>\n\n\n\n Google has more info on the Web Rendering Service (WRS)<\/a>, which includes things like denying permissions, being stateless, flattening light DOM and shadow DOM, and more that is worth reading.<\/p>\n\n\n\n Rendering at web-scale may be the 8th wonder of the world. It\u2019s a serious undertaking and takes a tremendous amount of resources. Because of the scale, Google is taking many shortcuts with the rendering process to speed things up. At Ahrefs, we are the only major SEO tool that renders web pages at scale<\/a>, and we manage to render ~150M pages a day to make our link index more complete. It allows us to check for JavaScript redirects and we can also show links we found inserted with JavaScript which we show with a JS tag in the link reports:<\/p>\n\n\n\n Google is relying heavily on caching resources. Pages are cached; files are cached; API requests are cached; basically, everything is cached before being sent to the renderer. They\u2019re not going out and downloading each resource for every page load, but instead using cached resources to speed up this process.<\/p>\n\n\n\n This can lead to some impossible states where previous file versions are used in the rendering process and the indexed version of a page may contain parts of older files. You can use file versioning or content fingerprinting to generate new file names when significant changes are made so that Google has to download the updated version of the resource for rendering.<\/p>\n\n\n\n A common SEO myth is that the renderer only waits five seconds to load your page. While it\u2019s always a good idea to make your site faster<\/a>, this myth doesn\u2019t really make sense with the way Google caches files mentioned above. They\u2019re basically loading a page with everything cached already. The myth comes from the testing tools like the URL Inspection Tool where resources are fetched live and they need to set a reasonable limit.<\/p>\n\n\n\n There is no fixed timeout for the renderer. What they are likely doing is something similar to what the public Rendertron<\/a> does. They likely wait for something like networkidle0 where no more network activity is occurring and also set a maximum amount of time in case something gets stuck or someone is trying to mine bitcoin on their pages.<\/p>\n\n\n\n Googlebot doesn\u2019t take action on webpages. They\u2019re not going to click things or scroll, but that doesn\u2019t mean they don\u2019t have workarounds. For content, as long as it is loaded in the DOM without a needed action, they will see it. I will cover this more in the troubleshooting section but basically, if the content is in the DOM but just hidden, it will be seen. If it\u2019s not loaded into the DOM until after a click, then the content won\u2019t be found.<\/p>\n\n\n\n Google doesn\u2019t need to scroll to see your content either because they have a clever workaround to see the content. For mobile, they load the page with a screen size of 411×731 pixels and resize the length to 12,140 pixels<\/a>. Essentially, it becomes a really long phone with a screen size of 411×12140 pixels. For desktop, they do the same and go from 1024×768 pixels to 1024×9307 pixels.<\/p>\n\n\n\n Another interesting shortcut is that Google doesn\u2019t paint the pixels during the rendering process. It takes time and additional resources to finish a page load, and they don\u2019t really need to see the final state with the pixels painted. They just need to know the structure and the layout and they get that without having to actually paint the pixels. As Martin Splitt<\/a> from Google puts it:<\/p>\n\n\n\nWhat is JavaScript SEO?<\/h2>\n\n\n\n
How Google processes pages with JavaScript<\/h2>\n\n\n\n
![\"pasted](\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2020\/06\/pasted-image-0.png\")
<\/figure>\n\n\n\n1. Crawler<\/h3>\n\n\n\n
![\"1.search](\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2020\/06\/1.search-console-crawled-as.png\")
<\/figure>\n\n\n\n2. Processing<\/h3>\n\n\n\n
Resources and Links<\/h4>\n\n\n\n
<link><\/code> tags. However, links to other pages need to be in a specific format for Google to treat them as links. Internal and external links need to be an <a><\/code> tag with an href<\/code> attribute. There are many ways you can make this work for users with JavaScript that are not search-friendly.<\/p>\n\n\n\n<a href=\u201d\/page\u201d>simple is good<\/a>\n<a href=\u201d\/page\u201d onclick=\u201dgoTo(\u2018page\u2019)\u201d>still okay<\/a>\n<\/pre>\n\n\n\n
<a onclick=\u201dgoTo(\u2018page\u2019)\u201d>nope, no href<\/a>\n<a href=\u201djavascript:goTo(\u2018page\u2019)\u201d>nope, missing link<\/a>\n<a href=\u201djavascript:void(0)\u201d>nope, missing link<\/a>\n<span onclick=\u201dgoTo(\u2018page\u2019)\u201d>not the right HTML element<\/span>\n<option value=\"page\">nope, wrong HTML element<\/option>\n<a href=\u201d#\u201d>no link<\/a>\nButton, ng-click, there are many more ways this can be done incorrectly.\n<\/pre>\n\n\n\n
Caching<\/h4>\n\n\n\n
Duplicate elimination<\/h4>\n\n\n\n
Most Restrictive Directives<\/h4>\n\n\n\n
3. Render queue<\/h3>\n\n\n\n
4. Renderer<\/h3>\n\n\n\n
![\"01](\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2020\/06\/01-javascript-seo.png\")
<\/figure>\n\n\n\n![\"3](\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2020\/06\/3-js-site-explorer.png\")
<\/figure>\n\n\n\nCached Resources<\/h4>\n\n\n\n
No Fixed Timeout<\/h4>\n\n\n\n
What Googlebot Sees<\/h4>\n\n\n\n
![\"02](\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2020\/06\/02-javascript-seo.png\")
<\/figure>\n\n\n\n