Introduction to Verdaccio - Trivago 2019
•Download as PPTX, PDF•
0 likes•83 views
The document summarizes a presentation given by Ayush Sharma and Juan Picado on Verdaccio, a lightweight private npm proxy registry. It outlines some of the problems with public npm registries like availability, control, security and affordability. It then introduces Verdaccio as a solution, describing it as a lightweight, private registry that acts as a proxy, caches packages, and has an optional configurable web interface. The presentation demonstrates Verdaccio's installation, use as a private registry, support for uplinks to other registries, and plugin capabilities.
Recommended
Recommended
More Related Content
What's hot
What's hot (12)
Similar to Introduction to Verdaccio - Trivago 2019
Similar to Introduction to Verdaccio - Trivago 2019 (20)
Recently uploaded
Recently uploaded (20)
Introduction to Verdaccio - Trivago 2019
- 1. A lightweight private npm proxy registry Ayush Sharma & Juan Picado trivago open-source conference Dec 2nd 2019
- 2. Juan Picado • Front-End Engineer at eBay • Maintainer at Verdaccio • Berlin 🇩🇪 • @jotadeveloper
- 3. Ayush Sharma • Software Engineer at trivago • Maintainer at Verdaccio • Düsseldorf 🇩🇪 • @ayusharma_
- 4. Plan - The problems - An introduction to Verdaccio ( by Juan ) - Live demo with Verdaccio - Q & A
- 5. Front-End Tooling Back-End What is a registry? All projects meet in the registry
- 6. Node.js Package Managers npm Inc Facebook, Google, Exponent, Tilde Zoltan Kochan
- 7. General workflow Public registry eg. NPM Inc Mostly we interact with public registries directly.
- 8. All packages come from the registry https://twitter.com/bitandbang/status/1134872073896169472
- 9. Concept of a private registry Public Remote(s)
- 10. You are missing a lot
- 11. The Problem(s)
- 12. Availability
- 22. The main registry might remove packages by mistake https://blog.npmjs.org/post/169432444640/npm-operational-incident-6-jan-2018
- 23. Control / Sharing / Integration
- 24. Your code is just small part of your Application https://twitter.com/liran_tal/status/10677753762298347
- 25. A modern application is highly modularized
- 26. https://twitter.com/housecor/status/1078634947831914496 97% of the code your app uses is shared
- 27. https://twitter.com/housecor/status/1078634947831914496 create-react-app has ~100k weekly downloads.
- 28. https://twitter.com/rauchg/status/807626710350839808 You need private registry to write integration tests.
- 29. Security
- 30. You might have made a mistake https://twitter.com/seldo/status/791061480904855552
- 31. You might have made a mistake https://blog.npmjs.org/post/101934969510/oh-no-i-accidentally-published-private-data-to
- 33. Self Hosted SaaS 💰💰💰 💰💰 SaaS uses volumed based pricing FOSS
- 34. Case Study “We ran the math, npm charges $7/customer/mo and every user has to have a paid account; verdaccio can effortlessly scale to hundreds of users and tens of packages a month” “We use it in production on a single DigitalOcean droplet, $5/mo” https://sheetjs.com/
- 35. Quick recap to problems - Availability - Control - Security - Affordability - Latency ( Not a big factor ) - Firewall ( In China )
- 36. A lightweight private npm proxy registry
- 43. • Lightweight Private Registry • Proxy (Uplinks) • Cache (Storage) • Entirely Optional Configuration • Pluggable Application • Web User Interface • MIT License What’s Verdaccio?
- 45. Docker and Kubernetes 6.600.000 pulls https://github.com/verdaccio/docker-examples
- 46. Private Registry All benefits of npmjs registry without sending the code to the public
- 47. @scopes and .npmrc It is hard to maintain and insecure
- 48. Uplinks Handle the distribution efficiently pnpm install --registry http://verdaccio.npm.tool
- 50. Storage can be hosted in the cloud Storage
- 51. Web Interface React, Typescript, Material-UI
- 52. Plugins Build plugin your own plugin https://verdaccio.org/docs/en/dev-plugins
- 53. E2E Tests Testing packages before publishing them
- 54. E2E Verdaccio helps avoiding mistakes https://twitter.com/adamlacombe/status/1084968234607144961
- 56. E2E
- 57. Keep in Touch • http://chat.verdaccio.org • https://opencollective.com/verdaccio • https://twitter.com/verdaccio_npm • https://github.com/verdaccio
- 59. Thanks
Editor's Notes
- We will start with the problem about registries in javascript ecosystem, then Juan will give you a brief introduction to verdaccio, if we get some time left in the end then we’ll give you a demo with Q&A.
- Before we start to talk about Verdaccio. I would like to talk about NPM registry. Does anyone know what a registry is? That’s right! Thanks! So a npm registry is a public collection of packages of open-source code for Node.js. It’s used by backend and Frontend developers!
- And...How can we install the packages that are in the registry? I am sure most of you use NPM right? Although NPM is the most used manager to install packages, there are a number of alternatives as Yarn created by group of companies and Pnpm, a very nice open source solution. So for you it doesn't matter which library to install your package, they will do the same thing: send requests to NPM registry. https://code.fb.com/web/yarn-a-new-package-manager-for-javascript/
- When we do install npm install, we mostly interact with public registries. Mostly we use npmjs as a default registry.
- But, What’s important here today the registry, IT IS WHAT MATTERS, nothing else, independently you are using as a client, all of then are useless without a registry. You can actually install tarballs by you own just having a registry available.
- We explain the flow that a request follows. The node management tool does a request. I.e npm / yarn / pnpm The registry receive the request and after compute the right remote try go get an updated resource. If it founds something updates the cache. Finally the registry responses the initial request Have you notice something in this flow? Client tools only works with one registry at the time. That is the power of registry. It can interacts with different remotes. It can point to npm / github package / yarn package registry all together.
- I find this tweet pretty interesting. Since npm started to be a trend, we based our application architectured to be based on one public registry and sharing has become a culture and this leads to some problems as the tweet says.
- Now, We’ll cover the most common scenarios and problems which can be solved by having a private registry.
- So the first major problem is availability. It means when you need something and that thing is not available to you.
- So you love travelling and you are in train. And you decide to work from train because you are a very committed developer.
- Then you go to your project directory and update everything.
- And you hit npm install in termnial.
- You get this beautiful error message in your terminal. After sometime you figure out that your internet is not working.
- Edward told me yesterday that you should have a cat in your presentations. So here the cat.
- Now, You turn on your personal hotspot and you connect your laptop with internet.
- And npm install again.
- And then other error message. You just realised that you are having a very rough day. And it says that NPM is down. Now you are stuck and can not work anymore.
- Since NPM tries their best keep the registry up but sometimes it fails. And you do not have any other option to wait for it working if you do not have a private registry.
- One unusual problem, but possible is this one: Last year...in 2018...some NPM employee by mistake removed some important packages from the registry...after that the users could not build their applications anymore. Now you can think how much business you can lose if you are directly relying on npm.
- Control means how much authorization you have on your packages. Sharing means what happened when you publish your packages and how they fulfill the compatibility or they integrate with other packages.
- Nowadays we work with really complex applications, composed with decens or thousands of dependencies as we can observe in the following graph. If we observe the content might be really hard to find your own code which usually represent no more than 3% of the whole bundle.
- If we have a zoom in into the bundle we can perfectly observe that we relies on modules, shared for others developers through a public repository.
- Now you can see that how much code you own and how much you are dependent on the public registries like npm, github, yarnpkg. And if they are down, you are down.
- Also, If you are directly publishing to public registries, you have some sort of responsibility. In the following tweet can you imagine if something happens in create-react-app that can put some businesses on risk. That’s why it is very very important to make sure integration before publishing to npm.
- In this world of move fast break things, we need some integration tests to make sure the compatibility with different versions and other packages in Javascript ecosystem. But the question is how can you write integration tests for package compatibility. First, You need to publish them. A private registry can help you to write some integration tests. We’ll cover E2E tests using verdaccio in later section.
- Another possible bad scenario: NPM has a two-factor authentication, but it’s not enable by default...so when you write the command: npm publish, in a matter of seconds the package will be published ... and you know ... once on the internet, always on the internet! Especially in the case of NPM that has many mirrors like yarn ... You can send an email to the support, they will write a nice disable description in the published package, but they can not do much. They can not unpublish your package.
- The registry run by npm Inc. is not the only npm registry. there are several thousand mirrors of the registry, run by various private individuals for their own purposes. When you publish any public package, it is replicated to all of these registries within 2-3 seconds.
- There are three types of alternatives present in the market. The final choice might depends of many variables only you know. But The self hosted is jFrog artifactory which can cost 30K $ per year. Sass is npm enterprise / github package registry which can cost around 7$ per user per month. But it really depends on your use case. But we’re here representing the FOSS and Verdaccio at this point, one of the best alternative for it. Becuase it is free. Just pay the server cost.
- Registry as a Business If you are running a business where distribute npm package is esencial you might look in to the experience of SheetJS. SheetJS (open source) has shared their experience and the result for them is a success. npm force that every user has to have paid account if you are willing to distribute through their SaaS platform Using a Droplet for just 5$ / month you can have a registry in production and save a money
- As my partner has explained, there are a lot of points of pain that many services cannot solve and here is the solution I purpose, Verdaccio.
- As my partner has explained, there are a lot of points of pain that many services cannot solve and here is the solution I purpose, Verdaccio.
- As my partner has explained, there are a lot of points of pain that many services cannot solve and here is the solution I purpose, Verdaccio.
- As my partner has explained, there are a lot of points of pain that many services cannot solve and here is the solution I purpose, Verdaccio.
- What’s is Verdaccio? The basic definition is a private registry where you can publish your packages It acts as a proxy, by default npmjs but you can hook as much you want and apply different rules to some of them It caches packages, here is the magic of the offline mode. Configuration is optional, the default one enough to getting start We allow plugins, verdaccio is small, you can make it big It has an beautiful UI MIT License to avoid any compatibility issue
- Install verdaccio is really simple, just install the package globally and you are set. If you are a wild developer willing to use the latest features use the “next” tag.
- We have an official Docker Image and is pretty straightforward to install. More than 6 millions of pulls by now are using in their projects and companies. It’s the most popular way to use Verdaccio. If you want to see more detailed examples, I’d recommend look in your Docker examples repository.
- Verdaccio provides all benefits of npm registry but with couple of differences: You keep the full control of it You can host private packages even if the are not scoped You can override public packages even if the name collides You can --force publish packages You can control access for publish and access packages Verdaccio does not publish on public registry, always is local.
- Disadvantages of this approach One scope cannot point two registries It does not accept Regular Expressions, Verdaccio does !! It’s hard to maintain and inscure
- Verdaccio has the advantage to route any package name, independently whether is scoped or not to a specific remote registry. One package can be fetched from multiple registries (as a fallback) Verdaccio handles the distribution efficiently
- Remote registries might be protected, for such reason the uplinks provides easy set up of HTTP Headers for Authentication. Add as much headers you need, no limit. Grab a token from some available environment variable Define token directly within the config file
- If you need to scale, then it’s more appropriate use a different sort of storage, for instance, an Amazon S3 bucket. Using this plugin will allows you to scale verdaccio in multiple processes delegating the persistence in the cloud.
- Web Verdaccio provides a nice UI for searching and browsing your private packages, we are working on improve it and add more features. We use React, Flow and Material-UI as primary tools for development.
- Share your plugin with the community We provide Types (in Flow and Typescript) to make this process smooth as possible Check our documentation for further information
- If you are in the mono repo wave, I totally recommend testing your packages before publish them. I cannot go into too much details but let me tell a small story.
- Here we have an co-creator of IONIC Framework, he was updating build scripts and in order to do not a mistake he turned off the wifi!! Smart guy !! But he might be smarter ! There is a better way to avoid mistakes. Using Verdaccio as a mock registry.
- This a list of project which are currently using Verdaccio as E2E testing.
- This a list of project which are currently using Verdaccio as E2E testing.