Snusmumriken
Snufkin, Włóczykij, Manrico, Wanderer in a forest of code
Saturday, 23 February 2013
I'm in New York Times
Well - my phone was published there.
You can find it by searching for Duke Theatre. A tiny typo on the ad. Someone swapped 2 digits.
Recently I'm getting more or less 3 calls a day. A surprising and quite amusing interruption while I work.
Imagine a dialogue:
Elderly gentlemen: Good evening, I'd like to make a reservation for 2 tickets for...
Me: Good evening, unfortunately you've called a wrong number. Please call...
G: But this number is in the Newspaper - answers gentlemen with a quite emphatic voice.
Me: I beg your pardon? Do you mind telling me which newspaper it was in?
G: It is in New York Times - says gentlemen with even more emphatic voice.
Moral of the story - don't trust newspapers.
Especially not - New York Times
:)
Wednesday, 6 February 2013
fabric low level socket error (vagrant)
Monday, 17 December 2012
Servname not supported for ai_socktype
Configuring service - this time Django Sentry - through puppet on EC2.
We wanted to have AWS mail backend for simplicity.
Pretty natural configuration. To achieve better server resources usage I've enabled gevent for sentry.
Thumbs up for fantastic tool sentry has became.
Though there was a strange glitch. When Sentry is receiving the exception for the first time - than it wants to send an email to the user.
What I was getting at this point was another internally reported exception:
From:
"sentry.plugins.sentry_mail.models._send_mail"
Exception:
"Error processing 'post_process' on 'MailProcessor': [Errno -8] Servname not supported for ai_socktype"
It was raised by:
gevent/socket.py in getaddrinfo
All seems good, no typo in config. nslookup works fine on smtp host name. Though there was one tiny glitch.
EMAIL_PORT = '587'
Apparently when port is passed as string than it's being resolved using entries /etc/services which is pretty awesome. So the quick fix was just to remove quotes:
EMAIL_PORT = 587
Idiotically simple solution for slightly strange exception and kind of cool to see socket errors on C level. Missed it for a while :)
Wednesday, 7 November 2012
IPVanish on OS X
Imagine you can test a service from any place in the world for one set price. Pretty useful.
Well, almost...
But, when selecting it I was getting "Unable to connect to..." alert.
Now it's something more reasonable. Why would I have an issue with kext load?
Let's see if I don't have any conflicting modules:
theta:~ munhitsu$ kextstat
124 0 0xffffff7f81dff000 0x6000 0x6000 foo.tap (1.0) <7 1="1" 4="4" 5="5">7>
125 0 0xffffff7f81e05000 0x6000 0x6000 foo.tun (1.0) <7 1="1" 4="4" 5="5">7>
126 3 0xffffff7f81e0b000 0x36000 0x36000 org.virtualbox.kext.VBoxDrv (4.1.23) <7 1="1" 3="3" 4="4" 5="5">7>
127 0 0xffffff7f81e41000 0x8000 0x8000 org.virtualbox.kext.VBoxUSB (4.1.23) <126 1="1" 35="35" 3="3" 48="48" 4="4" 5="5" 7="7">126>
128 0 0xffffff7f81e49000 0x5000 0x5000 org.virtualbox.kext.VBoxNetFlt (4.1.23) <126 1="1" 3="3" 4="4" 5="5" 7="7">126>
129 0 0xffffff7f81e4e000 0x6000 0x6000 org.virtualbox.kext.VBoxNetAdp (4.1.23) <126 1="1" 4="4" 5="5">126>
That's it. foo.tap/tun seems strange to me. Quick search - it used to be installed by old versions of Tunnelblick. Gotch ya! Let's remove Tunnelblick and... the same.
Unfortunately removing Tunelblick bundle does not remove modules.
The proper way to do it is to:
let's disable modules:
theta:~ munhitsu$ sudo kextunload -b foo.tun
theta:~ munhitsu$ sudo kextunload -b foo.tap
let's remove them from filesystem:
theta:~ munhitsu$ sudo rm -Rf /Library/Extensions/tun.kext
theta:~ munhitsu$ sudo rm -Rf /Library/Extensions/tap.kext
And now IPVanish works perfectly with OpenVPN. Quick glimpse to the logs:
(...)
Nov 7 10:23:20 theta.local com.ipvanish.helper.openvpn[10851]
SUCCESS
Tuesday, 14 February 2012
pip install django-dowser
So don't chase your memory leaks using print and scratching your head.
Instead...
1. Issue:
pip install django-dowser
2. Add django_dowser to settings.py:
INSTALLED_APPS = (
#...
'django_dowser',
#...
)
3. Modify urls.py:
urlpatterns += patterns('',
(r'^dowser/', include('django_dowser.urls')),
)
That's all folks
;)
Friday, 22 July 2011
Making django a better WSGI citizen
Aka Monkey patching Django with paste registry
Once upon a time there were two Django projects. A legacy one and a newly developed one. The modern one was developed independently as a separate project. They were both working on separate databases (in fact Django db backend was not used, it was just mongodb and redis backends), using different set of middlewares, different context processors… Two healthy projects.
What is also important they were both using gevent and deployed as wsgi servers.
During production deployment of the modern one we’ve decided that if we can make them both to work together using wfront, than we can save on amount of python instances being run on the same cluster (read: RAM).
In this configuration one instance of wfront wsgi app wraps couple of wsgi apps within one thread. Thread is than shared by gevent managed requests dispatch.
+-----+ +-------------------------------------+ | | | | | +--->| Legacy Django Project as a WSGI App | | | | | | | +-------------------------------------+ | | | W | +-------------------------------------+ | | | | | F +--->| Modern Django Project as a WSGI App | | | | | | R | +-------------------------------------+ | | | O | +-------------------------------------+ | | | | | N +--->| Other WSGI App 1 | | | | | | T | +-------------------------------------+ | | | | ... | | | | +-------------------------------------+ | | | | | +--->| Other WSGI App X | | | | | +-----+ +-------------------------------------+
Idea is simple and as long as we talk about clean WSGI apps that are not cluttering globals than we are fine.
Unfortunately Django is using globals. And the first one I’ve reached was parsed settings object.
The standard way to read settings from within Django project is to use:
from django conf import settingsWhat it does is a creation of a lazy/proxy/cache singleton that once called loads all settings.py content to itself. Location of settings.py is taken from os.environ['DJANGO_SETTINGS_MODULE']. Thanks to it we have one namespace optimizing access for all Django apps/modules to take settings from.
What we need it to have a separate settings.py file loaded for each of our Django based wsgi apps. Certainly we have to do something about loading the settings from file name defined in os.environ['DJANGO_SETTINGS_MODULE'].
So we need to hijack following entry:
class LazySettings(LazyObject):
def _setup(self):
try:
settings_module = os.environ[ENVIRONMENT_VARIABLE]
if not settings_module: # If it's set but is an empty string.
raise KeyError
except KeyError:
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." % ENVIRONMENT_VARIABLE)
self._wrapped = Settings(settings_module)
settings = LazySettings()Than we need to make it configurable, preferably on initialization/call to Django wsgi application created by application = django.core.handlers.wsgi.WSGIHandler().
I wanted to keep surgery on minimal level so inspired by great article by Simon Willson about djng I’ve decided to use paste.registry module. It’s a nice way to have a request-specific (thread safe) objects. Greenlet level safety was already maintained by geven monkey patching.
import gevent.monkey; gevent.monkey.patch_all();
The idea behind paste.registry is to wrap existing WSGI app with registry setup middleware. Than you can use other WSGI middleware to inject some data, then another to edit it and finally wsgi app to read it. Data is passed using request object, but data access api is perfect for monkey patching of existing code.
+------------+ +------------+ +-------------+ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | WSGI | | WSGI | | WSGI | | | | | | | | Middleware | | Middleware | | Application | | | | | | | | Registry | | B | | | | Manager | | | | | | | | | | | Request | | | | | | --------------------------------------------->| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Response | | | | | | <---------------------------------------------| | | | | | | | +------------+ +------------+ +-------------+
Let’s start from monkey patching of django.conf to go through registry defined object.
def monkey_patch_conf():
import django.conf
django.conf.settings = StackedObjectProxy(name="patched Settings")Next step is to properly initialize the settings object using an extra middleware. Fortunately there is already a way to initialize Settings from specific file. Just Settings(settings_module).
class SettingsMiddleware:
def __init__(self, app, settings_module):
self.wrapped_app = app
self.settings_module = settings_module
def __call__(self, environ, start_response):
from django.conf import Settings, settings
real_settings = Settings(self.settings_module)
if environ.has_key('paste.registry'):
environ['paste.registry'].register(settings, real_settings)
return self.wrapped_app(environ, start_response)A final step is to initialize registry and wrap it all together.
server = WSGIServer(address, wfront.route([
('localhost', RegistryManager(SettingsMiddleware(application, "foo.settings")), None),
('*', RegistryManager(SettingsMiddleware(application, "bar.settings")), None),
]))Paste RegistryManager initializes the paste registry. Than our SettingsMiddleware loads appropriate settings file and than in the end django application works on the request with having appropriate settings loaded.
+------------+ +------------+ +-------------+ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | WSGI | | WSGI | | WSGI | | | | | | | | Middleware | | Middleware | | Application | | | | | | | | Registry | | Settings | | Django | | Manager | | | | | | | | | | | Request | | | | | | --------------------------------------------->| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Response | | | | | | <---------------------------------------------| | | | | | | | +------------+ +------------+ +-------------+
And that is al folks. At least for first global :) Next global was optimization for context_processors which patching required pretty the same steps.
Code Sample
import gevent.monkey; gevent.monkey.patch_all();
from gevent.wsgi import WSGIServer
from paste.registry import RegistryManager, StackedObjectProxy
import wfront
import logging
log = logging.getLogger(__name__)
def monkey_patch_conf():
log.debug("monkey_patching django.conf")
import django.conf
django.conf.settings = StackedObjectProxy(name="patched Settings")
class SettingsMiddleware:
"""
A middleware that loads settings from a specific file and exposes its content to whole wsgi thread, coroutine.
Requires:
- RegistryManager middleware
- monkey_patch_conf
exemplary usage: RegistryManager(SettingsMiddleware(wsgi_application, "foo.settings"))
"""
def __init__(self, app, settings_module):
self.wrapped_app = app
self.settings_module = settings_module
def __call__(self, environ, start_response):
from django.conf import Settings, settings
real_settings = Settings(self.settings_module)
if environ.has_key('paste.registry'):
environ['paste.registry'].register(settings, real_settings)
return self.wrapped_app(environ, start_response)
def main():
monkey_patch_conf()
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
address = "0.0.0.0", 8000
server = WSGIServer(address, wfront.route([
('localhost', RegistryManager(SettingsMiddleware(application, "foo.settings")), None),
('*', RegistryManager(SettingsMiddleware(application, "bar.settings")), None),
]))
try:
log.info( "Server running on port %s:%d. Ctrl+C to quit" % address)
server.serve_forever()
except KeyboardInterrupt:
server.stop()
log.info( "Bye bye")
if __name__ == "__main__":
main()
Thursday, 21 July 2011
Having issues with Xcode 4.1 upgrade on Lion?
Should be simple, but surprisingly while you install the upgrade of xcode you are stuck on iTunes not closing properly.
Well in fact you can close it but xcode installer seems to miss this fact.
You ask yourself what's going on?
Does it sound familiar?
The reason is simple. Even though you close iTunes (BTW: why can't xcode installer do it by itself?) there is still iTunesHelper in background blocking Xcode installer.
So all you have to do is to open Terminal and issue:
$ ps -ef | grep iTunes
501 406 96 0 12:49AM ?? 0:00.14 /Applications/iTunes.app/Contents/MacOS/iTunesHelper.app/Contents/MacOS/iTunesHelper -psn_0_180268
Have fun :)
PS: That's something new for Apple to have QA issues.

