Thursday, February 26, 2009

Django without cookies but putting the sessionid in the URL

Django sessions require cookies

Django is written to assume that its client is a browser, and that its browser supports cookies. The "How to use sessions" document makes it clear (here) that this is an intentional design decision that use should be well aware of, especially if you want to break those rules as described here. The reasons for that decision are clear and correct, but were not good for our uses.

The Django cookie requirement was not good for our project

But in our own project, a community-based online radio network called YiqYaq, which must support a wide range of client types, I found more and more reasons to want to use cookies. These reasons were:
  • Our clients were often not browsers, but different tools. We provide a simple url-based API to most functionality and forcing those clients to understand and support cookies too much work on the client's shoulders. I.E. we want to make our API as easy as possible so more people will want to use it.
  • A surprising number of early testers (friends who we asked to preview our site and services) where on browsers that did not support cookies for random URLs, usually because that was IT corporate policy at their work, but sometimes because well-meaning any-malware supported that policy.
  • We were moving do Django from a tomcat implementation, and missed the convenence of having jsession id inserted and used (almost) automatically whenever cookies were not supported.
So, for our project, we did the near-equivalent of tomcat's "jsessionid" for our web pages, and for our non-web clients expect them always to use jsessionid.

Our implementation of "jsessionid" Django version 0.96.2

Note: applies to Django version 0.96.2 -- I've no idea what other versions this works with -- I also have not idea whether this is the best way to accomplish what I wanted, but it's the way that works for me

I won't describe our utility functions for how we do the equivalent of tomcat's encodeURL, so that the sessionid is in the URL whenever cookies are not supported. That is a fairly trivial issue, and is bound to be different on each site. This article is about the primary problem, which is how to pass a string to the client to represent a session, how to get that string back from the client, and how to turn that string into a django session.

1) Create the session_key magic value

Once the authentication has happened, in standard django ways, we use this code to generate the "session_key" string


from django.contrib.sessions.models import Session
from django.contrib.sessions.middleware import SessionWrapper
...
...
...
request.session = SessionWrapper("")
auth.login(request, authuser)
obj = Session.objects.get_new_session_object()
session_key = obj.session_key
new_session = Session.objects.save(session_key, request.session._session,
datetime.datetime.utcnow() + datetime.timedelta(seconds=(10 * 60 * 60)))
# good for 10 hours


The session_key is then passed on to the client as the magic value to represent this session.

2) Create the session_key magic value

When the client contacts the server, and is not uing cookeis, it must then pass back the magic session_key value to the server as a parameter in its URL (or however else you want to pass it). Turning that magic string into a session so that all your other django code can use it normally is trivial:

request.session = SessionWrapper(sessionid)

what is less trivial is how to write data back to the session, in the same was as the cookie middleware would do it, so that changes to the session are preserved along with the magic cookie string. In other words, the following code must be applied before your server returns to the client or else all the changes it may have made to the cookies will be lost:

# sessions are written to work with browser cookies, but we don't want to work
# with browswer cookies - so fake it

# this code is copied from django.contrib.session.middleware - the difference is
# this code ignores cookies

# If request.session was modified, or if response.session was set, save
# those changes and set a session cookie.
try:
accessed = request.session.accessed
modified = request.session.modified
except AttributeError:
pass
else:
if modified:
print "MODIFIED!!!"
if request.session.session_key:
print "### USE OLD KEY ###"
session_key = request.session.session_key
else:
print "### CREATE NEW KEY ###"
obj = Session.objects.get_new_session_object()
session_key = obj.session_key

print "### AND SO ON ###"
new_session = Session.objects.save(session_key, request.session._session,
datetime.datetime.utcnow() + datetime.timedelta(seconds=(10 * 60 * 60)))
# good for 10 hours



Good luck

That's it. I hope it works well for you. It's been working well at YiqYaq so far. And if you have better ways to do this I hope you let me know.

Tuesday, February 24, 2009

Converting fom Screenflow .mov to Flash video .flv for our web site.

Over the weekend I did my first screencast demo for our YiqYaq web-radio site, using Screenflow on Mac OS X. The result was a big .mov file that had a couple of problems 1) it was too darn big, and 2) I could find lots of browser clients that would not play that .mov format. Quick research showed that close to 100% of my audience could play flash videos.

So my problem became, how to turn the Screenflow .mov file to a .flv playing on our web site.

The first approach was to upload it to YouTube. YouTube accepted the video, and converted it smoothly, but the final results was squashed down 360 pixels tall, which is too small to effectively show our demo. (See the too-small youtube version here).

After lots of trial and error, I did finally get to an acceptable embedded flash video .flv file, that I can embed on our web site. Maybe there's a better approach, but these are the steps I took:

How I turned Screenflow into embedded Flash vide on our own web site (as seen here):

1) Record and edit with Screenflow

I recorded the interraction of my browser along with audio. The browser window happened to be set at 778x950 (widthxheight). Editted the result, and did all the other demo stuff.

2) Screenflow Export

Selecting Screenflow Export I chose these settings:
  • web high best quality
  • scale 74% to 576x702 (because 702 is short enough to fit on most computers yet tall enough to represent my screencast well, and also because 576 is divisible by 16--which turns out later to be important--and 702 is even; if these requirements had not been met I would have used a tiny bit of letterboxing to make sure they were matched)
  • enabled motion blur
  • change frame rate to 30 fps
  • change vide settings to optimize for streaming
  • change audio settings for best quality
3) Compress and prepare with Stomp

Stomp is software to compress video. It costs a few dollars ($29). For my purposes I didn't actually use it to compress so much as to reorder the audio/video so that the next phase (ffmpegX) doesn't have the jerky problems it gets from the way the Screenflow outputs audio in .mov files.

For Stomp I used these settings always chooseing best quality versus lowest compression (which actually resulted in a larger file that what I started with, but I think that is taken care of later):
  • Make my own preset
  • 30 fps
  • multipass
  • best compressor quality
  • audio best
4) Convert to .flv with ffmpegX

ffmpegX is a minimal wrapper around a lot of open-source video software. It's $15 shareware. You can build all those video convert tools from the open source, but for me ffmpegX was worth it because, even though it was an effort to install, it was a whole lot less effort than installing all tools it's made from.

From ffmpegX open the Stomped file and output to FLV format. I used these options:
  • From options selected "High Quality" and "Two-pass encoding"
  • video size 576x702
  • framerate 15 (which is half the framerate 30 used in steps up to this point)
  • play around with the video bitrate field until you get a value that ffmpegX shows up as green--sometimes it is slow to show this so you need to switch on different tabs to force it to update--in my case I ended up with video bitrate 1400
5) AddFLV metadata with FLV MetaData Injector

The file that results from ffmpegX does not have the metadata (especially for height and width) that the next step (JW Player) is going to need to display the video correctly (without the right metadata it will try to change the video to fit standard sizes). To perform this step I downloaded the free windows tool "FLV MetaData Injector" and run the executable ("flvmdi.exe") from a parallels session. There are other FLV metadata injector source tools you can use and build, but this was the quickest way there that I found.

The result of this step is a .FLV file suitable for any flash player.

6) Create web page and embed on web page with JW FLV Media Player

Many users can play the FLV files directly, but they won't look very elegant. It's also not very difficult to find a lot of FLV players, or even to roll your own. But I was in a hurry and JW FLV Media Player did everything I wanted (at a cost--open source and free for non commercial but €39 for a commercial license on one web domain)

Longtail provides very clear instructions for how to use their tool. Remember to set the width and height to the video size of the .FLV file you've been making, and to add at least 20 pixels to the height if you're going to use their control bar (which is the default).


That's it. Like I said, it may not be the best approach but it's the one I stumbled into and it worked for me (as seen here).