Adopting Docs-as-Code at Pinterest

[

](https://medium.com/@Pinterest_Engineering?source=post_page---byline--4f18ad169c25---------------------------------------)

Jacob Seiler | Software Engineer, Internal Tools PlatformJay Kim | Software Engineer, Internal Tools Platform

Charlie Gu | Engineering Manager, Internal Tools Platform

Technical documentation is the backbone of any successful engineering team, yet at Pinterest, it has consistently been a source of frustration. Despite using popular web-based wiki tools and experimenting with well-known alternative applications, our internal developer surveys reveal that documentation remains a top pain point. The issues boil down to two critical areas: quality and discoverability. Traditional solutions, such as doc-a-thon sprints or passionate appeals from senior leaders, have failed to produce lasting improvements. In 2021, we decided it was time to try something new.

We began exploring different approaches to enhance our documentation tools and processes, with a particular focus on the docs-as-code strategy. This initiative aimed to not only elevate the quality of our technical documentation but also to transform the culture of documentation at Pinterest. The result was PDocs, our internal documentation system.

Docs-as-Code

Docs-as-code is the philosophy that you should be writing documentation using the same tools and processes you use for code. In practice, this means several things:

• Documentation is often written in a markup language such as Markdown.
• Documentation is checked into source control often next to the code it is documenting. As a result, you use the same tools you use to author code such as source control and code review tools.
• There might be CI/CD processes involved in validating and shipping your documentation.
• You use a static site generator to turn your documentation into an HTML site.

We felt adopting a docs-as-code strategy would help solve our documentation problems. In summary, we can create a system that encourages good documentation practices just by using it. More specifically:

• Collaboration is accounted for. Many of the tools we were using make it easy to update docs but provide no tools around proposing changes and iterating on feedback.
• Quality control of docs becomes a lot easier. If you’re working on a developer platform, service, or library that is consumed internally, the documentation is essential and you want creative control. In previous wiki style tools that we’ve used, documentation would often be edited without consulting the owner. This has meant the developer experience you had originally authored in the docs may turn into something you no longer recognize. Code review tools ensure the right people are reviewing any changes made to docs.
• Discoverability is improved. Docs can be stored next to the code they are documenting. We can also index the docs into our intranet search providers.
• Documentation is more likely to exist and be up-to-date. Engineers can evolve code alongside documentation in the same PR. Documentation writing becomes a part of the development process rather than something you do after you’ve shipped a feature. If documentation is not updated or does not exist, the PR can be blocked.
• Design issues are caught earlier. By evolving the docs alongside code, you get an early sneak peak of developer experience by reading the docs. Often, usability gaps in the APIs you are trying to ship only reveal themselves once you realize how awkward the documentation is.
• Style is separated from content. Doc authors can focus on the content and the static site generator can generate docs that look professional and consistent. Mix both style and content such as in tools that use WYSIWYG editors. They can be frustrating to use as you sometimes need to wrestle with the underlying markup to get the whitespace exactly how you want it and it can distract you from writing good content.

Static Site Generators

To implement the docs-as-code vision at Pinterest, we had to adopt or create a static site generator that could take Markdown files and generate a site that people could visit to view their documentation. There are many popular open source documentation tools out there like Docusaurus by Meta and MkDocs by Tom Christie that are great at creating documentation sites for single projects. However, we did not want to set one up every time a new doc project needs to be created. This would involve setting up project boilerplate, installing dependencies, and setting up a CI/CD pipeline. Additionally, we believed that providing engineers with a standardized static site generator would help maintain consistency and reduce potential tech debt in the future.

Figure 1. Conventional static documentation site creation process using separate pipelines

To minimize the fixed cost of setting up a documentation project, we wanted a static site generator that could automatically colocate documentation projects from various file paths and repositories and generate a single centralized doc site. We wanted a developer experience where you could drop a simple config and Markdown file in any repository and have it show up in our centralized doc site once you’ve merged the change. We couldn’t find any existing open-source documentation tools that could do this easily so we ended up building our own — PDocs (Pinterest Docs, not to be confused with the python documentation tool pdocs).

Figure 2. Integrated static site generation using PDocs, which automatically scans repositories to create a centralized documentation site

Having a single centralized documentation tool allows us to improve the experience for everyone, such as with utilizing GitHub API integrations or UI improvements. Also, we can easily have every doc synced to search providers to make them discoverable.

Building PDocs

In line with the docs-as-code vision, PDocs is designed to allow documentation to be hosted across multiple repositories. The first step of the build process was to crawl each repository and search for PDocs projects.

Taking inspiration from MkDocs’s mkdocs.yaml, a PDocs project contains a pdocs.yaml. In the yaml, we describe the page tree as well as additional metadata that will help us render the documentation.

After cloning, we can use the pdocs.yaml config to determine which file paths to parse and render to HTML. We use a set of packages from the Unified ecosystem of utilities to process the Markdown ASTs and map them to React components from our UI library Gestalt. We built all of this logic into Next.js to handle static rendering at build time.

We also wanted a CLI experience for our new documentation tool. Something that we could use to easily scaffold out a new PDocs project with some basic config and templates:

We needed a dev server that would allow us to preview and auto reload our documentation as we’re iterating on its contents:

To achieve this, we had to introduce our own custom server and wrap Next.js to delegate React rendering to their Node.js APIs. In our custom server, we have logic that ensures the Next.js preview mode cookie is set for all requests made to the dev server. This tells Next.js to live render the requested HTML or JSON instead of looking for HTML or JSON that had been previously statically rendered at build time. This ensures our users can refresh the site to see a preview of their changes.

To get auto reload working, we introduced a server-sent event API endpoint that the doc site would automatically connect to. When reading the pdocs.yaml config, we initialize file watchers for each doc. If a doc changes, we push an event to the client and perform a “soft reload” using Next.js’ client router. This reloads the page component props instead of reloading the entire HTML document from the server.

PDocs UI

This is what the home page of PDocs looks like:

Figure 3. Home page of PDocs

We designed the UI to be project-centric so that people could discover what they needed by first finding the appropriate project. In our internal wiki, there isn’t always a clear way to organize and categorize documents. As a result, people often place documents in random locations. Consequently, our mental model of the wiki has become a messy heap of documents rather than a collection of well-organized trees. This has made using the search functionality the main way to enter docs, which exacerbates the discovery problem.

To prevent readers from encountering incomplete documents that could clutter the home page or search results, PDocs projects can now be marked as ‘published’ or ‘draft’. By default, when projects are created using our CLI, they are set to ‘draft’ in the pdocs.yaml file. This setting hides the project from the home page unless the ‘draft’ filter is selected in the UI and also prevents it from being indexed in our internal search engine. This approach is crucial to maintain reader trust, as encountering numerous unfinished documents can lead to frustration and potentially harm the success of a documentation project.

To improve navigation, we’ve added features like a “favoriting: option and a “recently viewed” section. As we keep growing the platform, we’re working on even more enhancements.

Figure 4. A look at a doc page within PDocs

In the actual docs page, we put a lot of thought into all of the UI surrounding the main content:

1. When viewing a doc project, we filter the search so that it only returns content for the project you are on. We found this to be helpful in providing more relevant search results. The user can always toggle back to global search if needed.
2. We have a header UI that contains useful metadata about the doc project you are browsing. Properties like the Nimbus project (which users provide in the pdocs.yaml), and last updated timestamp (which we pull from GitHub APIs), helps to make docs feel more trustworthy. Nimbus is the system Pinterest uses internally to track ownership of engineering entities and is where we can derive the correct Slack channel to show in the header UI. Our users can click on the Nimbus project to see the appropriate owners of the doc in case they need to contact them directly.
3. We generate a page tree on the left hand side based on the pdocs.yaml file. When using our wiki tool to read docs, we noticed the page tree was not always being utilized since it is not always apparent that it exists as it is minimized by default for many users. Having the page tree always visible helps with discoverability and encourages authors to be mindful of how they organize docs. (Note: for this blog post, some of the nav items have been minimized).
4. On the right hand side, we generate a table of contents for each page from the heading outline of the doc. Again, this helps with discoverability and organization.
5. We include some useful links that will open the docs in GitHub. We wanted to reduce the friction of making changes to docs. Clicking on the ‘Edit doc’ link will open the current doc in the GitHub web editor, which allows users to create a commit and pull request without leaving their browser. Because the docs are in standard Markdown, changes can also be previewed using GitHub’s Markdown renderer.

GenAI

Utilizing internal company knowledge with LLMs to better guide people on problems has become very popular in recent years. PDocs documentation has emerged as the best source for this due to its use of Markdown. Compared to other WYSIWYG documentation platforms, PDocs uses Markdown, which is the language best understood by LLMs. This eliminates the need to translate documentation between the human-written version and the one ingested by the LLM. Furthermore, because PDocs is custom-built, we can easily inject additional steps in the CI/CD pipeline to push the documentation to knowledge store providers, ensuring that updates are live.

We have developed several integrations with GenAI. There is now a chat option inside PDocs that allows you to ask AI questions directly while reading the documentation. We have also connected PDocs with our internal communication apps, so our AI Bot can automatically share helpful knowledge from PDocs whenever your team needs it.

Has Documentation Improved at Pinterest?

We initially limited PDocs adoption to word-of-mouth, so we had a small group of enthusiasts using the tool. This allowed us to focus on building features tailored to a small group of early users, creating loud evangelists who loved what we were building. In early 2022, we announced PDocs to the rest of the engineering org and we’ve seen tremendous adoption of PDocs from all corners of the engineering org. We currently have 140+ doc projects from 60+ GitHub repos written by 80+ teams, and PDocs has become one of the most visited internal sites at Pinterest.

While usage is important, the primary goal is to measure document quality. We’ve observed improved satisfaction in internal surveys and received positive feedback, indicating that people are pleased with the documentation. Qualitative feedback has been overwhelmingly positive, suggesting that writing and browsing documents in PDocs is a much better experience than using our existing wiki-style tools. We are continually exploring ways to enhance and measure document quality. One thing to note is that PDocs mainly features new documentation projects, which may explain the perception that PDocs documents are more maintained and higher quality. This makes it challenging to compare with our existing tools, which have accumulated legacy issues over time. However, early indications suggest that our investment in PDocs is paying off, supporting our thesis that poor tooling contributes to poor documentation at Pinterest, and that docs-as-code is the way forward.

Future

Enhancing documentation quality and implementing user-requested features to build trust and engagement remain our top priorities. Based on user feedback, we are focusing on two strategic areas:

1. Simplifying the experience for documentation writers to create and edit documents to increase efficiency.
2. Enabling better interactivity and visibility between readers and writers of documentation to improve long-term maintainability.

For the first point, last year, we introduced a Wiki to PDocs converter that enabled teams to migrate their documents into Markdown code with a single click. This resulted in a 20% increase in the number of documentation projects over two months. Moving forward, we plan to extend this capability to platforms such as Google Docs, allowing teams to draft their write-ups using other tools before moving them to PDocs. Furthermore, we plan to develop a live Markdown editor directly in PDocs, enabling teams to make quick adjustments on the fly and automate the PR creation process, drastically simplifying the experience when making minor edits to the documentation.

Regarding maintainability, we aim to introduce features like a health dashboard for document owners, which will provide insights into the quality and status of documentation, ensuring it remains accurate and up-to-date. Additionally, we are committed to incorporating user-requested features such as comments, which will foster greater engagement among our users.

Acknowledgement

We want to thank the ITP team (Jacob Seiler, Phil Saam, Hindreen Bag) for their work in building and maintaining the tool. We also want to express our gratitude to our leadership, Jooseong Kim and Chunyan Wang, for their sponsorship in making it happen. Additionally, we thank our writing team partners, Tierney Anthony, Benj Klang, and Kathryn Jones, for collaborating with us on setting standards for documentation. Lastly, we extend our thanks to our engineering partners, such as Zhanyong Wan and Jordan Culter, for promoting the wider adoption of PDocs.