The content below is taken from the original (OpenStack Developer Mailing List Digest May 14 to June 17), to continue reading please visit the site. Remember to respect the Author & Copyright.
SuccessBot Says
- Qiming: Senlin has completed API migration from WADL.
- Mugsie: Kiall Fixed the gate – development can now continue!!!
- notmyname: exactly 6 years ago today, Swift was put into production
- kiall: DNS API reference is live [1].
- sdague: Nova legacy v2 api code removed [2].
- HenryG: Last remnant of oslo incubator removed from Neutron [3].
- dstanek: I was able to perform a roundtrip between keystone and testshib.org using my new SAML2 middleware!
- Sdague: Nova now defaults to Glance v2 for image operations [4].
- Ajaeger: First Project Specific Install Guide is published – congrats to the heat team!
- Jeblair: There is no Jenkins, only Zuul.
- All
Require A Level Playing Field for OpenStack Projects
- Thierry Carrez proposes a new requirement [5] for OpenStack “official” projects.
- An important characteristic of open collaboration grounds is they need to be a level playing field. No specific organization can be given an unfair advantage.
- Projects that are blessed as “official” project teams need to operate in a fair manner. Otherwise they could be essentially a trojan horse for a given organization.
- If in a given project, developers from one specific organization benefit from access specific knowledge or hardware, then the project should be rejected under the “open community” rule.
- Projects like Cinder provide an interesting grey area, but as long as all drivers are in and there is a fully functional (and popular) open source implementation there is likely no specific organization considered as unfairly benefiting.
- Neutron plugin targeting a specific piece of networking hardware would likely given an unfair advantage to developers of the hardware’s manufacturer (having access to hardware for testing and being able to see and make changes to its proprietary source code).
- Open source projects that don’t meet the open community requirement can still exist in the ecosystem (developed using gerrit and openstack/* git repository, gate testing, but as an unofficial project.
- Full thread
Add Option to Disable Some Strict Response Checking for Interoperability Testing
- Nova introduced their API micro version change [6]
- QA team adds strict API schema checking to Tempest to ensure no additional Nova API responses [7][8].
- In the last year, three vendors participating in the OpenStack powered trademark program were impacted by this [9].
- DefCore working group determines guidelines for the OpenStack powered program.
- Includes capabilities with associated functional tests from Tempest that must pass.
- There is a balance of future direction of development with lagging indicators of deployments and user adoption.
- A member of the working group Chris Hoge would like to implement a temporary waiver for strict API checking requirements.
- While this was discussed publicly in the developer community and took some time to implement. It still landed quickly, and broke several existing deployments overnight.
- It’s not viable for downstream deployers to use older versions of Tempest that don’t have these strict response checking, due to the TC resolution passed [10] to advise DefCore to use Tempest as the single source of capability testing.
- Proposal:
- Short term:
- there will be a blueprint and patch to Tempest that allows configuration of a grey-list of Nova APIs which strict response checking on additional properties will be disabled.
- Using this code will emit a deprecation warning.
- This will be removed 2017.01.
- Vendors are required to submit the grey-list of APIs with additional response data that would be published to their marketplace entry.
- Long term:
- Vendors will be expected to work with upstream to update the API returning additional data.
- The waiver would no longer be allowed after the release of 2017.01 guidleine.
- Short term:
- Former QA PTL Matthew Treinish feels this a big step backwards.
- Vendors who have implemented out of band extensions or injected additional things in responses believe that by doing so they’re interoperable. The API is not a place for vendor differentation.
- Being a user of several clouds, random data in the response makes it more difficult to write code against. Which are the vendor specific responses?
- Alternatives to not giving vendors more time in the market:
- Having some vendors leave the the Powered program unnecessarily weakening it.
- Force DefCore to adopt non-upstream testing, either as a fork or an independent test suite.
- If the new enforcement policies had been applied by adding new tests to Tempest, then DefCore could have added them using it’s processes over a period of time and downstream deployers might have not had problems.
- Instead behavior to a bunch of existing tests changed.
- Tempest master today supports all currently supported stable branches.
- Tags are made in the git repository support for a release is added/dropped.
- Branchless Tempest was originally started back in Icehouse release and was implemented to enforce the API is the same across release boundaries.
- Tags are made in the git repository support for a release is added/dropped.
- If DefCore wants the lowest common denominator for Kilo, Liberty, and Mitaka there’s a tag for that [11]. For Juno, Kilo, Liberty the tag would be [12].
- Full thread
There Is No Jenkins, Only Zuul
- Since the inception of OpenStack, we have used Jenkins to perform testing and artifact building.
- When we only had two git repositories, we have one Jenkins master and a few slaves. This was easy to maintain.
- Things have grown significantly with 1,200 git repositories, 8,000 jobs spread across 8 Jenkins masters and 800 dynamic slave nodes.
- Jenkins job builder [13] was created to create 8,000 jobs from a templated YAML.
- Zuul [14] was created to drive project automation in directing our testing, running tens of thousands of jobs each day. Responding to:
- Code reviews
- Stacking potential changes to be tested together.
- Zuul version 3 has major changes:
- Easier to run jobs in multi-node environments
- Easy to manage large number of jobs
- Job variations
- Support in-tree job configuration
- Ability to define jobs using Ansible
- While version 3 is still in development, it’s today capable of running our jobs entirely.
- As of June 16th, we have turned off our last Jenkins master and all of our automation is being run by Zuul.
- Jenkins job builder has contributors beyond OpenStack, and will be continued to be maintained by them.
- Full thread
Languages vs. Scope of “OpenStack”
- Where does OpenStack stop, and where does the wider open source community start? Two options:
- If OpenStack is purely an “integration engine” to lower-level technologies (e.g. hypervisors, databases, block storage) the scope is limited and Python should be plenty and we don’t need to fragment our community.
- If OpenStack is “whatever it takes to reach our mission”, then yes we need to add one language to cover lower-level/native optimization.
- Swift PTL John Dickinson mentions defining the scope of OpenStack projects does not define the languages needed to implement them. The considerations are orthogonal.
- OpenStack is defined as whatever is takes to fulfill the mission statement.
- Defining “lower level” is very hard. Since the Nova API is listening to public network interfaces and coordinating with various services in a cluster, it lower level enough to consider optimizations.
- Another approach is product-centric. “Lower-level pieces are OpenStack dependencies, rather that OpenStack itself.”
- Not governed by the TC, and it can use any language and tool deemed necessary.
- There are a large number of open source projects and libraries that OpenStack needs to fulfill its mission that are not “OpenStack”: Python, MySQL, KVM, Ceph, OpenvSwitch.
- Do we want to be in the business of building data plane services that will run into Python limitations?
- Control plane services are very unlikely to ever hit a scaling concern where rewriting in another language is needed.
- Swift hit limitations in Python first because of the maturity of the project and they are now focused on this kind of optimization.
- Glance (partially data plane) did hit this limit and mitigated by folks using Ceph and exposing that directly to Nova. So now Glance only cares about location and metadata. Dependencies like Ceph care about data plane.
- The resolution for the Go programming language was discussed in previous Technical Committee meetings and was not passed [14]. John Dickinson and others do plan to carry another effort forward for Swift to have an exception for usage of the language.
- Full thread