Приятного просмотра

Pulp 3: Ready for a Test Drive

Опубликовано: 1 месяц назад
35 просмотров
👎 1
Скопируйте и вставте на Ваш сайт


by Austin Macdonald
At: FOSDEM 2019

Pulp is a platform for managing repositories of software packages, and its plugin architecture allows management of any type of content. Pulp’s web API enables users to synchronize, upload, and publish versioned repositories with simple but flexible workflows. This presentation will build on last year’s talk by Dennis Kliban 0, and will highlight simplified installation and deployment, better performance, new features, and a growing collection of plugins including rpm, python, docker, ansible, chef, ruby, file, and more in development. Pulp 3 is approaching maturity and is ready for you to test drive.
Room: UB2.252A (Lameere)
Scheduled start: 2019-02-02 16:00:00+01


All right welcome everyone for those of you who are new thank you very much for coming our next talk is Austin McDonald's and I'm thinking he's talking about poultry I'm gonna randomly guess enjoy hello everyone thank you for coming and especially thank you for coming this afternoon in this weather maybe this is not so terrible for you.

Guys but I am from the southern United States and this is not this is not for me so I appreciate braving the weather so today I'm gonna be talking about the advancements that we've made with poultry especially since last year in case any of you came to Dennis Cleveland's talk we've made a lot of improvements most of them have been under the hood.

Except for we've written a lot of new plugins for poultry and we're very pleased to be approaching our release candidate coming up so just in case some of you are not familiar with pulp 3 what is it I'm just gonna go through some of the basics if your production environments are created by installing software from reprise repositories that you do not control then you do not.

Really have full control over your production environments the entire thing to go down somebody could update something or remove left pad or whatever and everything could just completely fall apart and so if you absolutely need to have control over your production environments you need to be managing your own repositories and if you want to be extra careful.

You should be life cycling those repositories promoting them from dev testing production just standard best practices and you can do this with both so what we have here are a plugin architecture and so pulp can manage any content type can pretty much do anything all it needs is a plug-in that does that content type where open source and community driven.

This has especially been paying off recently we've been putting more and more effort into our mailing list and helping out the people to help us and we've been getting more and more contributions from the community including some plugins so we're very pleased about this of course Python is awesome but I think the main reason that it's fantastic that pulp is written in.

Python is because it allows us to leverage libraries like Django and Django rest framework okay so pulp will clearly solve all of your dependency problems all of your nervous things that keep you up at night so now I'd like to talk about some of the new things that you can expect from pulp 3 so remember this slide this is the plugins slide every single plug-in.

Here has a plugin for pulp 3 so you can manage docker Python chef Debian Ruby rpm and ansible roles so if you do any of those things you can do it with both so new features the whiz-bang new feature is version repositories as you can see here you can test it before you send it to production and I if I were a sysadmin I would be so nervous if I didn't.

And of course version repositories also gives us the opposite with pulp 3 you can roll back to a previous version and you do not have to republish so you can do this instantly and fix something if something goes wrong and of course this will happen because all software is terrible right ok this is a bit more of a subtle feature but plugins can now implement dynamic web api's so if you.

Have a fancy client that needs to do some negotiation like the docker client you can now talk with pulp directly and implement a full web api if you're up on the new hotness with Python then you know that async i/o is awesome and performant and everyone should be using it if you're not you should check that out but all you really need to know is.

That pulp 3 is faster deferred downloading is a pretty fun fun feature we also we lovingly call it lazy sync this was around for pulp too but only for the RPM plugin but basically if you've got a sink down of I don't know 50 thousand rpms I have no idea how many rpms are in the Eppler posit or II but I know it takes like all day and it will use up your entire disk so if you don't.

Want to do that what you can do instead is sink down only the metadata do it quickly and then fetch the bits as you need it it's very convenient and at in poultry lazy sink is going to be available to all the plugins that's built into the plugin API and lastly our REST API Doc's are auto-generated because we never like to update our REST API Docs so now we.

Don't have to another pain point for pulp too is that it was fairly difficult to install there's a lot of components you've got a database cue pulp itself and we've taken care of this by providing ansible roles and all you have to do is provide your own playbook and customize it and add variable season to paste this is how simple it is to choose which plugins you install just add a.

Couple of lines of the ml this is how simple it is to and do a source install just give a pointer to the source directory and that's pretty much how the ansible installer does for a whole pile of things and a note for the users you will know that the ansible installer will work because it's also the same way that the developer the developers are installing our stuff so.

We use it every day and if it breaks it's more annoying to us than it is to you I bet there's been a few improvements under the hood in pulp 3 most of this was covered last year in Venice Cleveland stuff but Postgres instead of Mongo RQ instead of celery somatically version plug-in API which is that's actually really fantastic even for the users because now you know.

Exactly which plugins can work with exactly which releases of pulp core which was somewhat of a pain point before no symlinks and we have a lot less code just for out of boredom I think one of the engineers on my team went through our repository last night and discovered that we're currently using 10% of the code that we were using for pulp 2 and we're approaching feature.

Parity and this is removing Doc's removing comments so 10% is pretty fantastic we're gonna be conservative and say after we get to parody will be 20% but we're still we're still very proud of that okay so this isn't a real demo this is just a slide demo because I don't trust internet ok so all your plugins in pulp 3 are gonna follow a very similar workflow and you're gonna.

Use the same api's so a docker demo will pretty much show you how you deal with ansible Jim chef Debian whatever so first you create a repository this is our fun little symbol for repository it's just a simple post of to the endpoint and you pass like one argument there's more options but I'm showing the minimal version second step you create a remote in this case the.

Remote is the docker registry and we're specifically pointing to docker hub busybox so what we're gonna be doing eventually is taking busybox from docker hub and then owning it ourselves so that docker hub could blow up and we'd be fine okay and then we kick off a sync task using our remote we think from the remote to the repository pulp goes and gets your.

Stuff next we create a publication a publication is simply just the metadata that a client is needs to use later and with docker this is actually very simple and lastly we create a distribution and now this is the very crucial step for sis admins who want to make sure that they are complete control of their environment notice in the dr. distribution that we've set the base.

Path here to testing what that means is that when you dr. pol you dr. pol from your host port then testing if I'd set this to production this would be flashed production this means that you are absolutely in control all the time you can keep updating your repositories without breaking the other people behind you and it creates a linear path of versions which you can you can have.

Production roll up to the next version if it works which makes all of this much simpler and just to prove that it works I did all of those things and then docker pulled from my sandbox environment did in fact work okay um that about wraps up the the high-level overview there are some relevant links here and if you would like to join us in pulp IRC.

Pound pulp pound pulp dev and our mailing lists are especially active so before I go to questions one thing that I would like to do is to direct you to these mailing lists specifically pulp list if you have any content types that you're looking for during the next period of pulps engineering we're gonna be in a release candidate for kind of a long time and during that time we're.

Going to be focused on plug-in development and so we need to know which plugins you would need to make your lives better so do it do we have any questions yeah haha that's a great question not yet but there will be before we release the GA yes both sinking from one pulp to another pulp yeah that is actually that's pretty.

Normal it's gonna have to be done at the plugin level so if we go back here when you create a publication you're creating the metadata that the client would use for most of the time but in some cases the Python plugin does this for instance we have to do a different kind of publish in order to publish metadata that could be read by another pulp but most plugins it just kind of works under.

The hood no worries but Python plugin it was not a big hassle to make it work yeah Oh of course um so pulp has split into many components of well not that many but we've got our web app we've got our content app they're separate and scalable horizontally we've also got a.

Task for a worker system that performed tasks from a few those can be scaled horizontally so if you're syncing a large repo and you want a lot of machines doing i/o or you've got like heavy calculations then that's how you would go about that you'd make a lot of workers so for 4h a what we're trying to do is we've got since we've got the content app split from the Web API that.

The content app can be controlled separately and scaled horizontally does that answer your question yeah yeah yeah so the default the default system is just your local file system but part of the reason why we were able to drop so much of the code is because we're leveraging great tools that were built on top of and in this case we get a lot.

For free from Django and Django's file system stuff we don't work yet on s3 but we will I'm told will we will be in the next two three weeks yeah what Sagan Oh instead of symlinks we're now just doing the the relative path is stored in a database and we're just acting like a normal Django web app but we were having problems with the send links with very.

Large installations we were running out of I know that's that seems like a ridiculously large ones me but you know just developers thinking the same thing every day yeah oh okay well we are existing in the same space as artifactory but we're free we're open source and in my opinion our plugins are more strong they are kept up-to-date by Red Hat engineers they.

Have Red Hat Kiwi behind it as well as a growing community so it depends on what you need to be fully honest but artifactory at the moment has more plugins than pulp but in the long term will see our plugin our plug-in API has made plug-in writing super easy by the way if any of you are going to config management camp we'll be doing a plugin writer workshop and you'll see that it's.

Really not that hard just writing a django up any other questions yeah yeah yeah so the question was is there filtered sink and the answer is that depends on the plug in it the sinks are completely written by the plug in writer and that's just by necessity we try to do some things under the hood to help them and we just really ended up getting.

In the way so the plug in writer writes to the sink task and so for instance the Python plugin does have some kind of a filtered stink you might specify that you want to sync only Django 111 plus and it will pull down all the versions of Gengo 1.11 plus so that's that's something that we do in Python I am not familiar with that and any of the other plugins except for it.

Will happen in docker pretty soon any other questions yeah are you asking about authentication so for the RC we're keeping our off very simple we have one user it's the admin user but you know if you're running a pulp and you are managing your repositories chances are you're at the admin if that's not your okay use case and you do need more stuff we one of the.

Reasons that we have not done it yet is that we have not had a very good understanding of how that would be used so if that does not if what we have right now it does not work for you now is absolutely the time to let us know because we can still slip in those backwards and compatible changes before the RC after that we will be stuck with it for kind of a long time any other.

Questions yeah sooner than sooner than I want to yeah so they we're planning on putting out our RC sometime very soon and then we will be having a very long our seat possibly in the months to happy air range will maybe more I don't know we're not gonna release it until it's right that's that's really what it comes down to and satellite will not include it.

Until it's right but they it we don't have to wait for all of the plugins to be perfect I think the plan is that maybe we'll be running helped to end pulp three side by side and some plugins will get ported one at a time so sorry I can't answer the time in some time frame any other yeah oh yeah so um the question is is there.

Going to be a GUI and I've actually heard this question more often today than I've heard in the past and so I have a suspicion that we probably will be doing the GUI and I talked with one of our UX designers about that today but it's very very tentative it's not the immediate priority right now the primary interface is the REST API but we will be having the CLI and we also.

Probably will be having a duty eventually if anyone likes writing gooeys our REST API is very easy to use okay any other questions all right well thank you very much you



Нет комментариев!