Sharovatov’s Weblog

Useful JS tips – getBoundingClientRect()

Posted in javascript by sharovatov on 12 June 2009

What do you use to get element’s offset? Looping through its parents and summing offsetLeft/offsetTop numbers? I bet you had problems with it. Well, I did, especially when additional positioning context was generated by an element with position!=static.

10 years ago in IE5.0 Microsoft invented a better way – getBoundingClientRect() method. It’s been popularised by PPK, specified in cssom-view spec, copied by Firefox 3,  Opera 9, Chrome 2, Safari; used in jQuery.

element.getBoundingClientRect() basically returns the object containing this element’s coordinates according to the window element. So you don’t have to do a loop, just call this method once and get the results. The only thing it doesn’t count for is scrolling offset, which can be calculated easily.

Extremely useful, thanks Microsoft IETeam!


Share :

Tagged with: ,

Exploring Windows Desktop Gadgets #4 – flyouts

Posted in IE8, javascript, windows 7 by sharovatov on 1 June 2009

This is the fourth post in "Exploring Windows Gadgets” series. In this post I’ll tell you about the flyouts.

As it’s practical to have small gadgets which don’t take much of screen space, it’s not possible to display much information on them. That’s why Gadget Platform provides a way to add a flyout object to your gadget.

Flyout theory

In theory, flyout is supposed to present additional or detailed information about anything you choose.

The great example of flyout use case is standard Windows 7 Stocks gadget – it shows basic stocks rates information in the main window and the if you click on the stock rate you’re interested, MSN Money chart for this stock rate is shown in a flyout. Here’s how it looks:

image

Flyout is a separate HTML page which you attach to your gadget similarly to settings window – by specifying the following:

System.Gadget.Flyout.file = 'flyout.html';

Another difference from settings dialog is that settings dialog is only accessible by clicking on a options button:

image

and flyout is shown/hidden programmatically by setting its show property to true/false:

//show flyout
System.Gadget.Flyout.show = true;

When you set show to true, flyout html will be rendered and its window will be automatically positioned depending on the content size and gadget’s position.

Flyout provides two events – System.Gadget.Flyout.onShow which is fired when the flyout is shown and System.Gadget.Flyout.onHide which is fired when the flyout is closed.

To set flyout content to something meaningful, you have to get access to flyout’s document from your gadget main window javascript. This is achieved by using System.Gadget.Flyout.document property – but the call to it will throw an exception if flyout is hidden, so make sure you either check System.Gadget.Flyout.show property for true or wrap the code which uses System.Gadget.Flyout.document in try-catch block.

Note that you can also access main gadget page from the flyout by using System.Gadget.document property which is always available.

Enough theory, let’s modify our Gadget to show only comments titles on the main page and render a flyout with comment’s content when title is clicked.

Practice

So here’s what I did:

  1. created flyout.html file
  2. referenced it as System.Gadget.Flyout.file = 'flyout.html'; in rss.js
  3. modified displayRSS function to store current comment object in paragraph’s expando property
  4. added toggleFlyout function to the rss.js file which passes the comment object from the <p>’s expando property to System.Gadget.Flyout.document and displays the flyout

The resulting files and the compressed gadget are here.

In the next posts I’ll show what styling options we’ve got in Gadgets Platform, will proceed talking about gadgets security and talk about advanced javascript techniques that can be used in gadgets.

Stay tuned! :)


Share :

Tagged with: ,

Exploring Windows Desktop Gadgets #3 – settings storage

Posted in IE8, javascript, widgets, windows 7 by sharovatov on 29 May 2009

This is the third post in Exploring Windows Desktop Gadgets series and it will be devoted to improving our comments feed reader gadget by adding settings storage functionality.

Settings storage theory

Any desktop program has its own settings storage (ini files, xml files, etc.) or uses a global OS storage (registry).

But in web-applications everything’s different – part of the functionality is provided by client (browser) and another part – by the server. Although it’s dead easy to store data on the server, on the client-side you can’t get access neither to file system nor to registry. (You actually can in IE, but this will require ActiveX scripting and therefore will ask user for an explicit permission.)

Until very recent DOM Storage specification draft, in most browsers you could only use cookies for storing some data on client side. However, IE5 introduced support for true client-side only storage – DOM Storage predecessor – userData. So it took almost eight years for the whole browsers industry to define and support similar client-side storage approach. And right at the time as DOM Storage is being drafted by W3C, it gets support in IE8.

The reason I’m telling you about DOM Storage in IE8 is that I have a strong feeling that this DOM Storage is used by Gadgets Platform to allow storing settings data for each gadget.

Settings storage practice

Settings dialog is a modal window which opens on top of your gadget. As it’s modal, the only way you can close it is to press OK or CANCEL button in its UI. Similarly to the gadget itself, settings dialog is a HTML page displayed by MSHTML.

As it would be useful to have comments feed URL and refresh frequency configurable, this is what I’ll add to my gadget:

image

In the settings dialog only area outlined in red is actually a HTML file, title and OK/CANCEL buttons are provided by Gadgets Platform:

image

To add the settings dialog to our gadget we’ll have to create a HTML file, let’s call it “settings.html”. To save comments feed URL and refresh frequency, we only need two fields and couple of labels. This is what I came up with.

If  you look at the JS code, you’ll see that it does two things:

  1. calls System.Gadget.Settings.read function to retrieve data from the Gadget settings storage
  2. defines a handler for System.Gadget.onSettingsClosing event

There’s a complete reference of Gadget platform events and properties on MSDN, but for our simple case it’s enough to say that System.Gadget.Settings.read reads object from the storage by its name, System.Gadget.Settings.write writes object to the storage with its name and System.Gadget.onSettingsClosing event fires when the settings dialog is closing. Settings store is separate for each gadget so don’t be afraid that your settings names can be accessed from other gadgets.

So in the window.onload handler of settings.html I try to get the data from the storage and set inputs values accordingly, and when the gadget is closing, I check if OK was pressed (e.closeAction == e.Action.commit) and if so, save inputs values into the storage. Dead easy.

Now we have to attach settings.html page to our gadget. This is done by adding the following javascript code to main.html file:

System.Gadget.settingsUI = "settings.html";

Now our Gadget will show this button in the control area:

image

And if you click on it, you’ll see your settings.html page rendered in a nice UI:

image

So now we have a settings dialog that reads and writes data from the Gadget settings store.  What we also need to do is to make our main.html read comments feed URL and refresh frequency from settings store as well. If you look at this javascript, you’ll see how it’s changed from the previous version. Basically, I just did the following:

  1. rss.js now doens’t have blog comments feed URL and refresh frequency hardcoded, instead, it reads these values from settings store:
    window.onload = function(){
    
      //read data from storage
      var data = getDataFromStorage();
    
      if ("" != data.url && 
          "" != data.frequency && 
          /^(\d*)(\.?)(\d*)$/.match(data.frequency)) startLoop(data);
      else document.body.innerHTML = "<b>Please make sure you"+
                                    "configured everything properly</b>";
    
    };

  2. System.Gadget.onSettingsClosed event handler is used to restart main loop with new settings:

    System.Gadget.onSettingsClosed = function(){
      var data = getDataFromStorage();
      startLoop(data);
    };

  3. And startLoop function has been rewritten to allow only one main loop (but that’s pretty irrelevant to Gadgets platform):

    var startLoop = function(data){
       //clear existing interval (if present)
       if ("undefined" != typeof arguments.callee.intervalHandler)
         window.clearInterval(arguments.callee.intervalHandler);
     
       //retrieve feed again
       getRSS(data.url);
     
       //set new main loop
       arguments.callee.intervalHandler = window.setInterval(
         function(){ getRSS(data.url); }, data.frequency * 1000
       );
    };

And here we go – gadget’s ready :)

Save it as rss-comments.gadget to your computer and install it – you will be able to set comments feed URL (so you can start using it to watch comments on your own blog) and refresh frequency in seconds.

In the next posts of this series I will cover flyouts, make the gadget look really slick with UI enhancements Gadget platform provides, show how debugging gadgets can be really effective and dive into more details of the Gadgets platform. Stay tuned for more interesting stuff!

Share :

Tagged with: ,

Googletranslateclient.com and Windows Accelerator Platform

Posted in usability, windows 7 by sharovatov on 21 May 2009

Yesterday I’ve blogged about Accelerators Platform and now I’m looking at a website googletranslateclient.com which offers to download a standalone VB6 win32 application that uses google translate service to dynamically translate a selected text.

Funny thing is that this program implements what Accelerator Platform is invented for, but in a very limited and counter-productive way – its functionality is limited to only one service so nobody can enhance it!

I think the best thing Microsoft could do would be to create a program that would capture text select event in any window and add Accelerators group to the context menu of that program!

Or alternatively, if this is too much work, just show Accelerators icon (accelerators-icon)  when any text is selected. I’m not much of a system application developer, so I’m not sure this is easy to do, but I’m sure people would benefit from it and developers wouldn’t need to add Accelerators API support in their apps!

Hope that someone will pick this idea.


Share :

IE8 Rendering Modes theory and practice

Posted in browsers, css, http, IE7, IE8, web-development by sharovatov on 18 May 2009

Many web-developers moan about the Rendering Modes switch that’s been introduced in IE8. I’ve got some thoughts on this.

Rendering modes switches history

As you may know, IE5 on Mac (followed by IE6 on Windows) introduced Quirks and Standards compliancy modes with a DOCTYPE switcher. All other browsers followed the move and introduced similar approach – they tried to mimic IE behaviour in quirks mode and tried supporting CSS as close to the W3C standard as they could.

DOCTYPE switch approach was really straight forward – if you put a valid DOCTYPE for your HTML page, browser switches to the standards compliancy mode; if DOCTYPE is omitted, quirks mode is used.

This switch was required because many sites were built at the time when even CSS1 was a draft, and for browsers to evolve safely (and add newer standards’ support) without breaking the existing web these old websites were to be rendered properly.

At the time the idea of using DOCTYPE as a rendering mode switcher was fine – almost all pages that had been built without standards support in mind didn’t have DOCTYPE. And if developers wanted to use modern standards, they added HTML or XHTML DOCTYPE and the page was rendered using standard-compliant parser.

Quirks mode and DOCTYPE switch locked old web with old rendering engine, IE5 engine.

I’m not wrong – after IE4 won first browsers war (BW I) nearly all sites were built for IE4 and IE5 only, so all other browsers had nothing to do but copy IE5 behaviour.

It’s important to note that DOCTYPE can only toggle standards compliancy on and off.

Therefore DOCTYPE switch only solves backwards compatibility issue and does nothing to prevent future compatibility issues.

When developers started using DOCTYPE switch, IE6 was a major browser and all the sites were build around its CSS support level. So when IE7 was shipped, it had better support for CSS2.1 but had the same DOCTYPE rendering modes switch.

That’s why many of these websites had rendering issues and had to fix CSS/JS for IE7 specifically. Conditional comments and conditional compilation came to rescue. However, with XP SP3 update IE6 started showing @jscript_version equal to that of IE7 and conditional compilation became pretty useless.

In IE7 old web-sites in quirks mode continued being rendered by IE5 rendering engine, and standards compliancy engine was just an updated version of IE6 standards compliancy rendering engine with some bugs fixed. And while IE7 was a minor update to IE6 but still caused so much trouble, one could imagine what IE8 with proper CSS2.1 support would do to websites. Basically, all the websites with DOCTYPES that were just fixed for IE7 Standards Compliancy mode would break in IE8 again.

Other browsers didn’t have a similar problem, and there’re many reasons to this. First of all, by the time when first IE7 beta was shipped, other browsers had really insignificant market share (so their developers could change anything and wouldn’t be blamed for “breaking the web”), were updated really often and were mostly installed by IT-related people who knew how to update the browser. So DOCTYPE rendering switch was and still is fine for non-IE browsers.

And Microsoft had to solve interesting problem in IE8 – support CSS2.1, support IE7 standards compliancy mode and support IE5 quirks mode. But DOCTYPE can only toggle standards compliancy on or off! They couldn’t drop support for very old but still existing sites build in quirks mode, couldn’t drop IE6 and IE7 support, and couldn’t leave IE8 without proper CSS2.1 support.

So we see that if any browser is locked at some version and this version gains noticeable market share, DOCTYPE switching will not work as vendors will have to choose what to support – older sites or newer features.

And Microsoft found a way – they gave web-developers and users a way to control in which mode sites are rendered.

IE8 Rendering Modes Theory

In IE8 there’re 6 rendering modes:

  • “Edge” Standard compliancy mode
  • IE8 Standards mode
  • “Emulate IE8” mode
  • IE7 Standards mode
  • “Emulate IE7” mode
  • IE5 – Quirks mode

Edge Standard compliancy mode basically tells the browser that it should use the most recent rendering mode – in IE8 it’s “IE8 Standards mode”, in IE9 it will hopefully be “IE9 Standards mode”.

IE8 Standards mode makes browser render a page using IE8 standards compliancy parser and renderer.

Emulate IE8 – DOCTYPE switch will be used to determine whether to use IE8 Standards compliancy mode or IE5 Mode.

IE7 Standards mode

makes browser render a page using IE7 Standards compliancy parser and renderer.

Emulate IE7 – DOCTYPE switch will be used to determine whether to use IE7 Standards compliancy mode or IE5 Mode. If you have a site that’s been built with IE7 Standards compliant mode support and you don’t want to change anything there, just add this META tag or HTTP header and IE8 (and all future versions of IE) will emulate IE7 behaviour, i.e. use IE7 Standards Compliant renderer when DOCTYPE is present and use IE5 mode when there’s no DOCTYPE. So you virtually “lock” your website to a current browser version behaviour. “Emulate IE7 mode” even sets User Agent string to IE7 string and report Version Vector as IE7 (so conditional comments will be evaluated as for IE7).

IE5 Mode – basically, it’s Quirks Mode. If we compare this to DOCTYPE switch approach, Edge mode is similar to a mode when DOCTYPE is set, and IE5 mode – to a mode when DOCTYPE is not present.

To explain modes in more details, I need to show how they are applied.

IE8 Rendering Modes Practice

As I’ve shown above, the whole idea of using DOCTYPE as a rendering mode switch failed because there were more than two types of documents on the web – those that should work in quirks mode, those that should work in IE7 standards mode, those that should work in IE8 standards compliancy mode and those that should work even with future versions of IE. So instead of two types we have three and a half :)

To give a proper way to control this, Microsoft and WaSP invented X-UA-Compatible http header, so you can either configure your server to send this header with corresponding value or add it as META HTTP-EQUIV to your page. For example:

<meta http-equiv="X-UA-Compatible" content="IE=8">

This declaration will tell IE8 to toggle IE8 Standards compliancy mode to render the page.

Here’s a list of modes:

Rendering mode X-UA-Compatible value
Edge Standard compliancy IE=edge
IE8 Standards compliancy IE=8
IE8 Emulation IE=EmulateIE8
IE7 Standards compliancy IE=7
IE7 Emulation IE=EmulateIE7
IE5 Quirks IE=5

 

IE8 Rendering Modes Switches Priority

First and main rule – if you specify X-UA-Compatible header or meta tag, IE8 will use the rendering engine you set.

If it’s EmulateIE7, DOCTYPE switch will be used to determine whether to use IE7 Standards mode or IE5 Quirks mode. See test case with DTD and without DTD.

If it’s EmulateIE8, DOCTYPE switch will be used to determine if it should use IE8 Standards mode or IE5 Quirks mode. See test case with DTD and without DTD.

If it’s IE=8, IE8 Standards Compliancy mode will be used regardless of presence or absence of DOCTYPE. See test case with DTD and without DTD.

If it’s IE=7, IE7 Standards Compliancy mode will be used regardless of presence or absence of DOCTYPE. See test case with DTD and without DTD.

If it’s IE=5, IE5 Quirks mode is used even if there is a DOCTYPE (actually it didn’t matter for IE5 if there was DOCTYPE or not – quirks mode was the only mode it had). See test case.

If it’s Edge, most modern rendering engine is used. For example, IE8 uses IE=8 mode regardless if there’s a DOCTYPE or not and IE9 will use IE=9. See test case with DTD and without DTD.

It’s worthy to note that X-UA-Compatible META overwrites X-UA-Compatible HTTP header, so you can set X-UA-Compatible HTTP header for the whole website and then have X-UA-Compatible META with a different value on some pages.

If website doesn’t have X-UA-Compatible META tag or HTTP header, IE8 uses EmulateIE8 mode i.e. DOCTYPE is used to switch between IE8 standards compliancy mode and IE5 Quirks mode.

This means that everybody needs to add to add X-UA-Compatible header or META to their websites.

Accepting the fact that web-developers wouldn’t update all the websites that were built around IE6/IE7 standards compliancy mode, Microsoft decision to take standards compliancy mode as a default would certainly “break the web”.

So Microsoft had to find a way around this which they successfully did by introducing a bunch of Compatibility View settings.

Compatibility View

As I’ve already said, when developer didn’t care about adding X-UA-Compatible header or META, IE8 works in EmulateIE8 mode and uses DOCTYPE switch to determine which rendering mode to use. If the site hasn’t got a DOCTYPE set, IE5 Quirks mode will be used, so backwards compatibility is not an issue. But if there’s a DOCTYPE, which Standards compliant mode to use? IE7? IE8? IE9 Standards Compliancy mode when IE9 it’s shipped? There’s no automatic way to choose.

And as IE8 Standards mode is a default one, there might be issues on sites that only supported IE6/IE7 Standard modes. Apparently, only user can see if the site he’s viewing has rendering issues. And Microsoft gave users an easy way to switch back to IE7 Standards Compliancy mode if required by clicking on a Compatibility View button (located to the left of refresh button).

image

IE8 will switch to EmulateIE7 mode and save the setting for the domain so all the pages from this domain will be rendered using IE7 engine.

Compatibility View button is shown only when X-UA-Compatible HTTP header or META is not set.

Also during IE8 install a user can specify if he wants to use Compatibility View List feature. Basically, when this feature is turned and user clicks Compatibility View button while browsing, IE8 reports to Microsoft that the website on this domain required user to press the Compatibility View button.

If statistics show that users tend to press Compatibility View button for the site, it’s included in iecompatdata.xml file which fresh copy is distributed with Windows Update. It can be viewed by typing res://iecompat.dll/iecompatdata.xml into the address bar.

So when Compatibility View List feature is enabled and there’s no X-UA-Compatible HTTP header or META, and DOCTYPE is present, IE8 will look if the site is in the res://iecompat.dll/iecompatdata.xml file and make Compatibility View button pressed by default.

As soon as the site gets X-UA-Compatible header or META tag, IE8 will use the rendering mode from its value and won’t even check Compatibility View List and also Compatibility View button won’t show at all.

Local Intranet security zone

Intranet differs from Internet a lot.

If Compatibility View List feature will work fine for public internet websites, Intranet sites should not be reported anywhere. So what rendering engine to use when there’s no X-UA-Compatible value and DOCTYPE is present? IE7 Standards Compliancy? IE8 Standards Compliancy?

Intranet a controlled environment where to decrease support costs IT pros usually set everybody with same set of software. As most successful companies use Microsoft technologies as a core of their IT infrastructure and because only IE has been providing a decent platform for building powerful intranet web applications, IE was always a de-facto standard browser in companies.

Many intranet web applications were targeting IE5.5/6.0 and nobody cared to refactor to get standards compliancy – why bother if there was always only one browser supporting a well-known list of technologies so cross-browser interoperability was never an issue? We can argue if this approach is good or not – but IE8 team had thousands of intranet applications they had to support. That’s why compatibility view was the only option for sites from Local Intranet security zone.

If the site is in the Local Intranet zone and there’s no X-UA-Compatible header or meta tag, DOCTYPE switch is used to determine which rendering engine to take – IE7 Standards compliancy or IE5 quirks mode.

If there’s no X-UA-Compatible value and site is in Local Intranet security zone, it will be rendered in EmulateIE7 mode by default.

Compatibility View button is not displayed at all.

But as soon as a developer adds X-UA-Compatible header or META, site will start working in the specified mode.

Developer Tools

With great set of developer tools that Microsoft provided for IE8 you can also view what rendering is being used and change it if required. This is useful for developers only and overrides any other settings. It’s very useful for testing – you don’t need IE7 in a virtual machine to test how a page looks in IE7 – just press F12 to get the developer tools and play with Browser modes or Document modes :)

Hope I managed to cover all the modes. In future posts I’ll write more on this topic.

P.S. As SelenIT pointed in comments, it’s very important that X-UA-Compatible meta tag must be put before any other elements in the head section, otherwise it might not work!


Share:

Windows 7 Desktop Gadgets

Posted in browsers, IE8, javascript, widgets by sharovatov on 14 May 2009

I’ve been playing with Windows 7 Desktop Gadget technology and I must tell you – I just love it! And I’ll tell you why :)

History

The approach of putting web-application right on a desktop was invented by Microsoft in 1994-95 (in Windows Nashville which was intended to be released in 1996) and was a part of Windows Desktop Update for Windows 95 and a core of Windows 98 Active Desktop. You could set an HTML page (plus JS, of course) as your desktop background and it would be run in IE. Microsoft even had a gallery of Active Desktop widgets. So web-widgets-like functionality was around almost twelve years ago. As with many other Microsoft inventions, it was way too ahead of time.

Another technology that played a role in inventing desktop gadgets is HTA – short HTML applications that run in a standalone way. You had to put your scripts (VBScript or JScript) and styles in a single HTML file, add a special description and rename the file extension to .hta. Here’s a sample code:

<html>
<head>
<title>HTA Test</title>
<HTA:APPLICATION
ID="objTest"
APPLICATIONNAME="HTA Test"
SCROLL="yes"
SINGLEINSTANCE="yes"
>
</head>

<SCRIPT type="text/javascript">
//some javascript code
</SCRIPT>

<body>

<!-- some html -->

</body>

You could access all ActiveX objects that your system provided – Scripting.FileSystemObject, UserAccounts, WMI etc. – basically everything that was supported in WSH and Script Runtime. If you were using classic ASP on the server, you could switch to writing HTAs with zero effort – language and all the objects would be the same. While any web-development IDEs could be used, there even was a special IDE for building HTAs – HTAEDIT.

So Microsoft had two nice technologies that weren’t actively used – Active Desktop as a way to integrate web-application with desktop and HTA as a core for writing standalone client-side web applications. Both technologies combined both to create Microsoft Desktop Gadgets – small standalone client-side web applications that run as dockable widgets on your desktop – pure HTML,CSS and JS packed with manifest file in an archive.

Usage

If you double-click on this archive, gadget will get installed and can be moved to any place of your desktop. IE8 core is used to run them, so you get full support for CSS2.1, great level of javascript and other IE8 goodness.

The gadgets concept was introduced In Windows Vista – but gadgets had to be positioned inside a special sidebar area – which is not very flexible. In Windows 7 you can put your gadgets wherever you want them to be.

Here’s a screenshot how gadgets look like on my desktop:

gadgets-screenshot[1]

Gadgets are especially useful when you have a widescreen monitor or two monitors configuration so that you’ve got much horizontal space, but even if you’ve got a normal monitor as gadgets can be freely positioned, you will find them useful.

I use Twitter, weather (standard), network traffic and cpu meter gadgets.

Awesome technology! I’m planning to build couple of my own gadgets and will definitely prepare a blog post on this :)

Update: It looks like Windows Mobile 6.5 will have support for similar gadgets functionality!


Share :

Apple submitted HTTP Live Streaming spec to IETF

Posted in http by sharovatov on 2 May 2009

As I’ve blogged recently, nearly a year ago Microsoft proposed an approach for adaptive video streaming over HTTP – Smooth Streaming. As Microsoft didn’t apply for a patent for this technology, I was hoping to see the same beautiful approach implemented in modules for other web-servers, or even as web-applications – as it’s really easy to implement.

The mistake Microsoft did was that they didn’t submit this technology standard to IETF to make it RFC – and that’s what Apple’s doing at the moment.

Yes, I’m not mistaken – Apple copied the whole idea, called it HTTP Live Streaming and submitted to IETF.

Yes, there’re differences, but they are absolutely insignificant:

  • Apple spec suggests extending M3U format for a playlist – Microsoft uses SMIL-compliant ISMC client-manifest file (i.e. playlist)
  • Apple spec defines that the server creates the playlist – in Microsoft approach the encoder creates the playlist
  • Apple spec defines encryption for media files – Microsoft doesn’t

And the whole specification that’s been proposed is weird – I think they just wanted to submit it as soon as possible before Microsoft Smooth Streaming approach gets popularity and becomes de-facto standard.

Here’s what jumped at me when I was reading the spec:

  1. section 6.2.3. Reloading the Playlist file – why specify the expiration time of the playlist separately when HTTP 1.1 already has flexible methods for
    setting expiration time of the resource?
  2. encryption – what’s the purpose of encrypting media files when there’s HTTPS? And if there’s a purpose – HTTP already provides a place where encryption could be "plugged in" – Transfer-Encoding, why didn’t Apple just
    register another transfer-coding in IANA?
  3. EXT-X-ALLOW-CACHE – why add this if HTTP already gives flexible tools to control caching?

So as I see it – Apple was just a little bit in a hurry to propose this “standard” – looks like they took Microsoft idea, added some proprietary bits and bobs without thinking them through, didn’t use what HTTP natively provides but bravely called the draft “HTTP Live Streaming”.

Awesome.


Share :

Silverlight smooth streaming and HTTP

Posted in http by sharovatov on 28 April 2009

I’ve read about smooth streaming technology, and I must say, I just love the way it works. It automatically and smoothly adjusts video quality and allows clients to view smooth video online regardless of their bandwidth and without a need to sit and wait staring at “buffering” message for ages – what it does it dynamically changes the video quality based on network bandwidth, CPU load and other factors.

It’s idea and implementation are so simple and beautiful that I wonder why nobody didn’t invent it earlier. This is what steps you have to follow to make it work:

  1. encode your video file in Expression Encoder 2 Service Pack 1
  2. upload it to IIS7 web server with IIS7 Smooth Streaming extension
  3. point your Silverlight player to the appropriate URL

That’s it. Here’s a showcase how this technology works – awesome!

Looks simple, right? It is simple, but there’s a huge amount of work hidden beside this simplicity. Let me dive into technical details a little bit :)

First of all, let me give you some background. Originally there were basically two types of streaming – stateful streaming and progressive download.

Good example of stateful streaming is RTSP. Client had to initiate connection to the server and send commands like PAUSE or PLAY, and server sent back video stream packets, client waited for its playback buffer to be filled with data and started playback. RTSP worked both over UDP and TCP (port 554 was used).

Progressive download – where client was sending traditional HTTP GET request and server responded with video data sent with use of HTTP chunked encoding, and client started the playback as soon as its playback buffer had enough data to play.

Both approaches had serious issues – RTSP couldn’t work for clients behind proxies or firewalls without extra efforts (that’s why Apple had to spend time inventing the wheel tunnelling RTSP over HTTP), progressive download couldn’t work fine for situation where bandwidth wasn’t good enough – have you had wonderful time sitting and staring at “Buffering” message?

So if you want to give highest video quality to users on a high bandwidth but still want to show users with low bandwidth at least something – you’ll create several versions of the same video and give users a way to choose and watch what they want.

But what if a user doesn’t know what bandwidth he’s got? What if the player itself could automatically select what video to download – high-res or low-res? What if the player could change bitrate during the playback if network conditions or CPU load changed? What if the player could instantly start playback from any point of the movie? And what if pure HTTP was used so that there would be no issues with proxies? What if each chunk of video could be perfectly cached by HTTP agent, such as proxy?

That’s precisely how Microsoft Silverlight Smooth Streaming works.

First of all, Microsoft decided to switch from their ASF format to MP4. There were many reasons for that, but the main point is that MP4 container specification allows content to be internally organised as a series of fragments, so-called “boxes”. Each box contains data and metadata, so that if metadata is written before the data, player can have required information about the video before it plays it.

So what does Expression Encoder do? It allows you to easily create multiple versions of the same video for different bitrates in this fragmented MP4 format. So you get up to 10 versions of the same video file with different resolution – from 320×200 up to 1080 or 720p. Each file internally is split in 2-seconds chunks, each chunk has its own metadata so you can programmatically identify the required chunk. Plus Expression Encoder creates two complimentary files (both follow SMIL XML standard) – *.ISM – server manifest file, which basically just describes to server which file versions have what bitrates; and *.ISMC, which tells a client what bitrates can be used and how many fragments files have.

Can you see the idea? IIS Smooth Streaming extension just maps URL to a chunk in a file. You do a HTTP GET request to a URL like this:

http://test.ru/mov.ism/QualityLevels(400000)/Fragments(video=61024)

And IIS Smooth Streaming extension checks “mov.ism” manifest file to find filename of the file with requested quality level (400000), opens and parses this file to get the chunk with requested time offset (61024). Then this chunk is returned to you in a normal HTTP response.

So you can query for any chunk of any one of your video files with the requested time offset.

Let me repeat it – you encoded your original video file into 10 fragmented video files with different bitrate. And you have a way to query for any chunk in any of these files.

So to play 10 seconds of video you have to do 5 consequent HTTP requests. As we have versions of the same video with different bitrate, we can get first chunk in the worst quality to see how it renders and what time it takes to download it, and then if CPU load is low and network is fast, we can query next 4 chunks with higher bitrate.

And that’s exactly what Silverlight Media Player component does – it requests chunk by chunk from the server and changes “QualityLevels” parameter in URL if conditions change. For example, if Silverlight Media Player sees that CPU load is too high and it’s dropping frames, or network becomes too slow, it changes “QualityLevels” parameter to a lower value so IIS Smooth Streaming extension serves next chunks from the smaller file with lower video quality.

Actually, when user starts the playback, first thing that Silverlight Media Player does is a request for ISMC file to find out how many different bitrate versions server has (and how to identify frames). And only then it composes URL to get the first chunk of video. Simple and beautiful technology.

So what do we have? Video plays smoothly – on old slow internet channels in lower quality and in full HD on fast internet and good CPUs. As HTTP is used as a transport – therefore no issues with proxies or firewalls; as each chunk is identifiable by a unique URL, every single chunk can be perfectly cached by proxies or other HTTP agents.

And as this technology is quite simple, there’s no doubt that there will be a similar module for other web servers, or even web applications achieving similar functionality!

Yes, as it’s encoded in multiple bitrate versions, it takes up to 6 times more space for one movie/clip, but if that’s what it takes to provide users with smooth playback in any conditions – I’m for it!

Thanks for another great technology, IIS Team!

Links:


Share :

Windows Live

Posted in windows live writer by sharovatov on 13 April 2009

I’ve just updated Windows Live Messenger to version 2009. It also offered to install other Windows Live tools, which I did. And you know what – Windows Live Writer rocks! Before I was using Microsoft Office 2007 Word for writing and publishing blogposts and it was doing it job, but now I’ve got a lightweight and really easy-to-use tool to manage my blog. And it’s free so I can install it on my laptop (which has got only Outlook 2007 on it). It can save any post as a draft, edit it later, it’s got live preview tool (which works amazingly fast), you can either work with wysiwyg or with plain html code – and I bet this is the cleanest wysiwyg I’ve ever seen. I really mean it – it generates absolutely clean HTML, and it uses my blog visual theme!

And the configuration process was great as well – I just entered my blog url, username and password, it fetched the theme information and that’s all – I’m writing, previewing and publishing posts!

So, the key features of Windows Live Writer (for me) are:

  1. it’s kerfuffle-free:
    1. it works fine with my wordpress blog
    2. wysiwyg generates really clean code so I don’t have to edit the code manually after I finished writing the blogpost
    3. it’s got good spellchecking mechanism
    4. it has a nice twitter plugin that puts a twit with a link to the new blogpost
    5. it contains most often used photo editing functions – so I don’t need to install anything else on my netbook to post to my blog when I’m travelling.
    6. it automatically fetches all the categories from the blog – you just need to set them, plus you can add categories if you want
    7. it can save drafts both to local storage and to the blog – so if I’m writing a huge blogpost, I can proceed editing it when I’m travelling or finish it at home.
  2. it’s lightweight and really fast
  3. it’s completely free
  4. and it’s got plugin architecture so we’ll see more interesting stuff coming!

Good stuff, Live Team, thanks!

P.S. for more information, read this blogpost.

Digg This
Tagged with: ,
Follow

Get every new post delivered to your Inbox.