The content below is taken from the original (OpenStack Developer Mailing List Digest May 7-13), to continue reading please visit the site. Remember to respect the Author & Copyright.
SuccessBot Says
- Pabelanger: bare-precise has been replaced by ubuntu-precise. Long live DIB
- bknudson: The Keystone CLI is finally gone. Long live openstack CLI.
- Jrichli: swift just merged a large effort that started over a year ago that will facilitate new capabilities – like encryption
- All
Release Count Down for Week R-20, May 16-20
- Focus
- Teams should have published summaries from summit sessions to the openstack-dev mailing list.
- Spec writing
- Review priority features
- General notes
- Release announcement emails will be tagged with ‘new’ instead of ‘release’.
- Release cycle model tags now say explicitly that the release team manages releases.
- Release actions
- Important dates
- Newton 1 milestone: R-18 June 2
- Newton release schedule [3]
Collecting Our Wiki Use Cases
- At the beginning, the community has been using a wiki [4] as a default community information publication platform.
- There’s a struggle with:
- Keeping things up-to-date.
- Prevent from being vandalized.
- Old processes.
- Projects that no longer exist.
- This outdated information can make it confusing to use, especially newcomers, that search engines will provides references to.
- Various efforts have happened to push information out of the wiki to proper documentation guides like:
- Peer reviewed reference websites:
- There are a lot of use cases that a wiki is a good solution, and we’ll likely need a lightweight publication platform like the wiki to cover those use cases.
- If you use the wiki as part of your OpenStack work, make sure it’s captured in this etherpad [9].
- Full thread
Supporting Go (continued)
- Continuing from previous Dev Digest [10].
- Before Go 1.5 (without the -buildmode=shared) it didn’t support the concept of shared libraries. As a consequence, when a library upgrades, the release team has to trigger rebuild for each and every reverse dependency.
- In Swift’s case for looking at Go, it’s hard to write a network service in Python that shuffles data between the network and a block device and effectively use all the hardware available.
- Fork()’ing child processes using cooperative concurrency via eventlet has worked well, but managing all async operations across many cores and many drives is really hard. There’s not an efficient interface in Python. We’re talking about efficient tools for the job at hand.
- Eventlet, asyncio or anything else single threaded will have the same problem of the filesystem syscalls taking a long time and the call thread can be blocked. For example:
- Call select()/epoll() to wait for something to happen with many file descriptors.
- For each ready file descriptor, if the file descriptor socket is readable, read it, otherwise EWOULDBLOCK is returned by the kernel, and move on to the next file descriptor.
- Designate team explains their reasons for Go:
- MiniDNS is a component that due to the way it works, it’s difficult to make major improvements.
- The component takes data and sends a zone transfer every time a record set gets updated. That is a full (AXFR) zone transfer where every record in a zone gets sent to each DNS server that end users can hit.
- There is a DNS standard for incremental change, but it’s complex to implement, and can often end up reverting to a full zone transfer.
- Ns[1-6].example.com may be tens or hundreds of servers behind anycast Ips and load balancers.
- Internal or external zones can be quite large. Think 200-300Mb.
- A zone can have high traffic where a record is added/removed for each boot/destroy.
- The Designate team is small, and after looking at options, judging the amount of developer hours available, a different language was decided.
- Looking at Designates implementation, there are some low-hanging fruit improvements that can be made:
- Stop spawning a thread per request.
- Stop instantiating Oslo config object per request.
- Avoid 3 round trips to the database every request. Majority of the request here is not spent in Python. This data should be trivial to cache since Designate knows when to invalidate the cache data.
- In a real world use case, there could be a cache miss due to the shuffle order of multiple miniDNS servers.
- The Designate team saw 10x improvement for 2000 record AXFR (without caching). Caching would probably speed up the Go implementation as well.
- Go historically has poor performance with multiple cores [11].
- Main advantages with the language could be CSP model.
- Twisted does this very well, but we as a community consistently support eventlet. Eventlet has threaded programming model, which is poorly suited for Swift’s case.
- PyPy got a 40% performance improvement over Cpython for a brenchmark of Twisted’s DNS component 6 years ago [12].
- Right now our stack already has dependency C, Python, Erlang, Java, Shell, etc.
- End users emphatically do not care about the language API servers were written in. They want stability, performance and features.
- The Infrastructure related issues with Go for reliable builds, packaging, etc is being figured out [13]
- Swift has tested running under PyPy with some conclusions:
- Assuming production-ready stability of PyPy and OpenStack, everyone should use PyPy over CPython.
- It’s just simply faster.
- There are some garbage collector related issues to still work out in Swift’s usage.
- There are a few patches that do a better job of socket handling in Swift that runs better under PyPy.
- PyPy only helps when you’ve got a CPU-constrained environment.
- The GoLang targets in Swift are related to effective thread management syscalls, and IO.
- See a talk from the Austin Conference about this work [14].
- Assuming production-ready stability of PyPy and OpenStack, everyone should use PyPy over CPython.
- Full thread