Our great system administrator amongst other sysadmin-specific posts published a really interesting post about deleting the flash plugin.

Проблема заключается в том, что хитрый установщик Flash, при установке дополнительно выставляет в ACL файлов информацию о запрете на запись (write) данных файлов для всех пользователей. Данное правило перекрывает все остальные права и не даёт удалить файлы в операционных системах считающихся с правами доступа NTFS. То есть для удаления достаточно зайти в свойства файла, на вкладке «Безопасность» (Security) нажать кнопку «Дополнительно» (Advanced) и удалить две строки описывающих запрет (Deny) на запись. После этого файлы удаляются без проблем.

For those who can’t read in Russian, here’s the essence excerpt:

When you try to delete flash plugin (flash6.ocx, flash10c.ocx) from %windir%\system32\Macromed\Flash folder, you get “permission denied” even if you’re the owner of the directory. The reason is that Flash plugin installer sets DENY WRITE permissions in NTFS ACL for this file, and DENY permissions rules always override ALLOW rules. So when you try to delete even you’re the owner of the files, you’re denied to do that :)

To fix this and delete the file, first run regsvr32 /u <path_to_file> command to unregister the file (if it’s registered in the system). Then you have to open file properties, go to “Security” tab, click “Advanced” button and remove two “Deny” entries there. Then you won’t have any problems deleting the file.

Thanks for sharing this, Dmitry!

When I have free time, I like to reorganise the knowledge I’ve got and prepare mindmaps/cheatsheets/manuals of interesting stuff. And the formal approach I usually use forces me to organise data in a way so that it won’t take me long to grasp the idea if I forget something.

And I also like posting resulting resources to blog — that’s a good English techwriting skills practice plus some publicity for the knowledge ;)

So this post is another one from the HTTP series and describes HTTP/1.1 persistent connections and chunked encoding.

HTTP/1.0 said that for every request to a server you have to open a TCP/IP connection, write a request to the socket and get the data back.

But pages on the internet became more complex and authors started including more and more resources on their pages (images, scripts, stylesheets, objects — everything that browsers had to download from the server). And for every resource request clients were opening separate connections, and it was taking time and CPU/memory resources to open a new connection, so from users prospective, resulting latency was becoming worse. Something could be done to improve the situation.

So HTTP IETF decided to implement a nice technique called “persistent connections”.

Persistent connections reduce network latency and CPU/memory usage of all the peers by allowing reuse of the already established TCP/IP connection for multiple requests.

As I mentioned, HTTP/1.0 client was closing the connection after each request. HTTP/1.1 introduced using one TCP/IP connection for multiple sequential requests, and both server and client can indicate that the connection has to be closed upon the completion of current request-response by specifying Connection: Close header.

Usually HTTP/1.1 client sends Connection: Close header with the last request in the queue to indicate that it won’t need anything else from the server, so that the TCP/IP connection can be safely closed after the request has been served with response. (Say, it wanted to download 10 images for the HTML page, it sends Connection: Close with the 10^th image request and the server sends the last image and closes the connection after it’s done).

Persistent connections are the default for HTTP/1.1 clients and servers.

And even more interestingly, HTTP/1.1 introduced pipelining support – a concept where client can send multiple requests without waiting for each response to be sent back, and then server will have to send responses in the same order the requests came in.

Note: pipelining is not supported in IE/Safari/Chrome, disabled by default in Firefox, leaving Opera the only browser to support and have it enabled. I will cover this topic in one of the next posts.

In any case, if the connection was dropped, client will initiate new TCP/IP connection and those requests that didn’t get a response back will be resubmitted through the new connection.

But as one connection is used to send multiple requests and receive responses, how does the client know when it has to finish reading the first request?

Obviously, Content-Length header must be set for each response.

But what happens when the data is dynamic or the whole response’s content length can’t be determined by the time transmission starts?

In HTTP/1.0 everything’s easy — Content-Length header can just be left out, so the transmission starts, client starts reading the data it’s getting from the connection, then when the server finishes sending the data, it just closes the TCP/IP connection, so client can’t read from the socket any more and considers the transmission completed.

However, as I’ve said, in HTTP/1.1 each transaction has to have correct Content-Length header because client needs to know when each transmission is completed, so that the client can either start waiting for the next response (if requests were pipelined), or stop reading current response from the socket and send new request through this TCP/IP connection (if requests are sent in a normal sequential mode), or close the connection it if it was the last response he was to receive.

So as the connection is reused for multiple resources’ content transmission, the client needs to know exactly when each resource download is completed, i.e. it needs the exact number of bytes it has to read from the connection socket.

And it’s obvious that if Content-Length can not be determined before the transmission starts, the whole persistent connections concept is useless.

That is why HTTP/1.1 introduced chunked encoding concept.

The concept is quite simple — if exact Content-Length for the resource is unknown at the time when transmission starts, server may send resource content piece by piece (so-called chunks) and provide Content-Length for each chunk, plus sends an empty chunk with zero Content-Length at the end of the whole response to notify client that this response transmission is complete.

To let HTTP/1.1 conforming clients know that chunked response is coming, server sends special header — Transfer-Encoding: chunked.

Chunked encoding approach allows client to safely read the data — it knows the exact number of bytes that are to be read for each chunk and knows that if an empty chunk arrived, this resource transmission is completed.

It’s a little bit more complex than HTTP/1.0 scenario where server just closes the connection as soon as it’s finished, but truly worth it — persistent connections save server resources and reduce whole network latency, therefore improving overall user experience.

Links and resources:

• HTTP/1.1 Persistent connections
• Wikipedia about HTTP Pipelining
• HTTP/1.1 Chunked encoding

Just noticed that when you parse an html file with DOMDocument’s method loadHTMLFile and there’s no DOCTYPE defined in your html, PHP will silently load an empty DOM document.

Just try saving the following in a test.html file:

<html><body><div id="toc">wtf</div></body></html>

And then run the following php code:

$doc = new DOMDocument();
if ($doc->loadHTMLFile('test.html')) {
  echo 'loadHTMLFile was successfully executed<br>';
  $toc = $doc->getElementById('toc');
  echo 'now trying to var_dump the $toc:<br>';
  var_dump($toc);
}

You’ll get NULL as a result of the var_dump call. As if getElementById couldn’t find the node.

Interesting?

Citing php.net,

The function parses the HTML document in the file named filename. Unlike loading XML, HTML does not have to be well-formed to load.

Does this imply that DOCTYPE may be omitted? I think so. But then the abovementioned code wouldn’t show NULL as a dump of $toc. Unfortunately, experiment shows that DOCTYPE is required, even a HTML5-ish
<!DOCTYPE html> will do the job. 

But why on earth doesn’t loadHTMLFile throw a warning or at least return false as it should according to the documentation? Nobody knows.

So if you notice that your DOM-based php script acts in a weird way, check if you have a DOCTYPE defined on the HTML document you’re trying to parse.

Hope this saves someone some time.

P.S. More bugs to come — if you have a HTML file saved in utf-8 codepage with BOM, loadHTMLFile will throw the following E_WARNING:

Warning: DOMDocument::loadHTMLFile() [function.DOMDocument-loadHTMLFile]: Misplaced DOCTYPE declaration in test-BOM.html, line: 1 in /home/test/www/test-DOMDocument.php on line 3

Remove the BOM and everything works fine. Apparently, loadHTMLFile doesn’t know that BOM usually indicates that the document is saved in UTF8/16/32. Weird.

P.P.S. Another issue. Try pointing loadHTMLFile to an HTML-document saved in UTF-8 with some international characters (Russian words, in my case). Then get a node with international characters and do echo $node->nodeValue. Are you getting corrupted symbols? I was. The whole project is in UTF-8, every single file is saved in UTF-8.

I added <meta http-equiv="Content-type" content="text/html;charset=utf-8" /> to the head section — characters started showing in a correct encoding, but the following WARNING appeared:

Warning: DOMDocument::loadHTMLFile() [function.DOMDocument-loadHTMLFile]: Input is not proper UTF-8, indicate encoding ! in /home/test/www/test-russian.html, line: 65 in /home/test/www/test-DOMDocument.php on line 29

And the only way to properly get rid of this warning is to add

<?xml version="1.0" encoding="UTF-8"?>

to the first line of your html document and it finally worked without any warnings or issues. Awesome. XML header must be used for loadHTMLFile to run properly. Way too buggy to use.

Share:

As Microsoft added live Twitter search results to BING search results page and Google promptly followed, Twitter becomes a very useful SEO tool which can bring additional traffic to your website. The main thing here is that the data is live – as far as I understand, search index is updated by Twitter directly, so the moment you tweet something, others will see it in the search results. Awesome!

Personally, I’m very afraid that Twitter will be seriously bloated with spam. And I don’t know yet how Microsoft and Google are going to filter out all the spam. Or maybe that’s something that Twitter will do? We’ll see.

In any way, Twitter will gain even more popularity and influence.

I’m finally back to blog. Finally started finding time between doing stuff at home, working at my great place of work and studying English :)

As you know, HTTP/1.1 spec said that conforming clients SHOULD NOT open more than 2 concurrent connections to one host. This was defined back in 1997 and at that time it seemed reasonable to have 2 simultaneous connections for a client, and noting that HTTP/1.1 introduced persistent connections concept, people thought that 2 simultaneously opened reusable TCP/IP connections would be enough for general use.

However, everything changes. Broadband internet came to mass market and people started thinking that better parallel download could benefit the whole website or a webapp perfomance. The history started with IE5.01, which was opening two connections by default, but there was a way to configure the number. So if you had a really good internet connection, you could make websites load significantly faster.

By the time IE8 development started, broadband connections became a standard for home internet, so IE8 started opening 6 connections (if the bandwidth allowed – on the dialup or behind a proxy it will still open 2). So IE8 engineers did a smart move and introduced the world with a browser that seemed to load sites faster.

Needless to say, Firefox 3 decided to change the value as well, so now Firefox 3 has 6 as a default value for network.http.max-persistent-connections-per-server configuration setting. Good for Mozilla for copying stuff from IE again!

And now HTTPBis team (Julian Reschke) commits the change which states that in the forthcoming HTTP standard the maximum amount of concurrent requests is not limited even with “SHOULD NOT” clause :)

Thanks HTTPBis team!

First of all, I started attending Muay Thai group here in Moscow. Three days a week, 2.5 hours each training. Found that 6 months boxing practice that I had several years ago gave me decent double jab, which is great. And it was also a surprise that I was able to survive the first warm-up – obviously, regular jogging and weights lifting did their job :)

I also took a two-week vacation to have proper rest and take time on sightseeing – still haven’t visited anything in Moscow which is a shame. Hope to spend this time productively and also finish all my hobby projects.

During the last weeks I noticed how many people in Twitter and around the world were interested in Iran elections. Once I spent enough time studying colour revolutions theory, which is a clever (and well-practiced) way to divide and rule principle that’s been working for ages. There’s enough information on the internet about this (you can start reading here or here). I really hope that all those people with green avatars in Twitter care to read a little bit about what the background is. The whole situation reminds me of how world went crazy about Tibet in 2008, but nobody cares about Tibet now :) And I’m pretty sure that there will be some protests against 2014 Winter Olympics in Sochi. Well, we’ll see.

Another interesting discussion I noticed on Twitter is about Outlook 2010 using Word engine to render and compose emails. fixoutlook.org shows us an interesting comparison screenshot of the same webpage displayed in Outlook 2000 and Outlook 2010. Yes, it’s a webpage, not an email. Would you spend hours on creating such email? I doubt. The only use case that I can see for such multimedia-intense email is for newsletters. So yes, Outlook 2010 will use the Word engine just as Outlook 2007 does – to create and edit emails. I’d love to be able to limit my emails styling to HTML4.1 and CSS 1. I don’t want javascript, flash, silverlight or anything else working in my email client. I simply want to read emails. Plain HTML plus some basic CSS would perfectly do for nearly any type of conversation. But of course, that’s my personal attitude.

There’s a couple links on the topic:

• howto-outlook.com has a good FAQ and explains why Microsoft took the decision to use Word engine in Outlook 2007
• reachcustomersonline.com gives a good review of HTML support  in different email clients and provides guidelines for creating good HTML email that will be displayed similarly everywhere

I also started exploring place where I live – found a river with ducks and ducklings and obviously fish (as there were two fishermen sitting fishing).

Here’re some photoes:

Opera announced “Opera Unite” concept – they integrated a web-server right into Opera and made Opera Desktop Gadgets run on it.

All the services Opera Unite offers are web gadgets, so they are built in html+javascript with some additional API provided by the browser.

To get any of the services, you must register at Opera. When you register, you give your computer a name, e.g. “home” and then you are provided with a URL home.yourlogin.operaunite.com where yourlogin is what you chose as a login when you were registering.

File sharing service is basically a web server directory listing exposed to the internet. This is my understanding how it works:

1. You point Opera Unite to a directory
2. Opera internal web server starts listening 8840 port locally
3. Opera opens a persistent connection to operaunite.com (213.236.208.30 IP address in my case)
4. So when anyone opens up home.yourlogin.operaunite.com, operaunite.com server requests the list of files from your machine using a persistent connection that you opened and sends the response back to user.
5. when you close Opera, web server is shut down, connection is dropped and nobody can download anything.

So sharing can work ONLY when your computer is working and Opera is running.

So none of the services can work when computer is turned off or Opera is not running.

When you want your sharing/chat/fridge services to be working, you will need to keep your Opera running. And if several users start using it, your computer will slow down significantly. And if you by any chance put a link to an image hosted in your Opera Unite on a popular site… Your computer will either stop responding or eat 100% resources.

That’s what John Resig, the author of beautiful jQuery says:

I just tried to visit six Opera Unite pages and only one resolved. The future of the web is two 9s: 0.99% uptime!

Useful service? I doubt.

Photo sharing service is just crap at the moment. My Opera Unite serving 1 client with a Photo Sharing page with thumbnails eats 60-70% of CPU and up to 200 Megabytes of memory. Full-blown web servers like IIS7 or Apache2 would serve this page and static files in a milliseconds without any noticeable resources eating. Thumbnails are created in really poor quality.

In Web Server service CGI is not supported, in-memory modules are not supported. PHP is not supported. The only language you can use is javascript. HTTPS is not supported.

Opera says that the communication between users is done directly. Truth is that it’s done through operaunite.com. Let me repeat it, all the traffic goes through operaunite.com. Are you ready to give all your information to Opera? Do you trust them so much? Do you care about your privacy? Do you think they will care about users after what they did to Windows 7 users in Europe?

Opera says this is a revolution – I can only see a bad (or alpha, not even beta) implementation of a rather poor technology. When I go out, I don’t leave my laptop working and Opera running, so the sharing won’t work. And I don’t want my browser to take 100% CPU and 400 Mb RAM when two users are watching static pages with static thumbnails. And it’s not p2p as all the traffic goes through Opera servers. There’re plenty of good services that do their work and don’t pretend to do a revolution where there’s clearly nothing revolutionary.

This makes me laugh:

Our computers are only dumb terminals connected to other computers (meaning servers) owned by other people — such as large corporations — who we depend upon to host our words, thoughts, and images. We depend on them to do it well and with our best interests at heart. We place our trust in these third parties, and we hope for the best, but as long as our own computers are not first class citizens on the Web, we are merely tenants, and hosting companies are the landlords of the Internet.

P.S. Opera engineers said that in the final version p2p file sharing will be implemented – well, let’s see.

Share :

]]> http://sharovatov.wordpress.com/2009/06/16/opera-revolution-fail/feed/ 13 vsh http://sharovatov.wordpress.com/2009/06/15/opera-to-reinvent-the-web/ http://sharovatov.wordpress.com/2009/06/15/opera-to-reinvent-the-web/#comments Mon, 15 Jun 2009 23:33:36 +0000 sharovatov http://sharovatov.wordpress.com/2009/06/16/opera-to-reinvent-the-web/ ]]>

Opera promises to reinvent the web tomorrow. I mean, really, reinvent the web? Invent TCP/IP, HTTP, DNS, HTML/CSS/JS, CGI, all other stuff that web consists of? I doubt.

Different sources speculate that it’s going to be something cloud-related. Maybe it will be a p2p-based traffic sharing with some clever algorithm to determine where to download the data from? But then I wouldn’t like anybody to be able to sniff my data. Maybe they will install a CDN to make useless Opera Turbo giving at least some performance benefit to users. Don’t know, we’ll see very soon.

But because of the last unfair actions that were taken by Opera against IE, I (from moral prospective) really hope that all the expenses they took for Opera 10 PR won’t significantly increase their market share. Competition should be fair, Firefox was able to get the same market share as IE in Germany without making users life harder. I really hope that lobbying expenses were higher than the income.

What do you use to get element’s offset? Looping through its parents and summing offsetLeft/offsetTop numbers? I bet you had problems with it. Well, I did, especially when additional positioning context was generated by an element with position!=static.

10 years ago in IE5.0 Microsoft invented a better way – getBoundingClientRect() method. It’s been popularised by PPK, specified in cssom-view spec, copied by Firefox 3,  Opera 9, Chrome 2, Safari; used in jQuery.

element.getBoundingClientRect() basically returns the object containing this element’s coordinates according to the window element. So you don’t have to do a loop, just call this method once and get the results. The only thing it doesn’t count for is scrolling offset, which can be calculated easily.

Extremely useful, thanks Microsoft IETeam!

Share :

As CNET states, Windows 7 will be shipped to Europe without IE8 because European antitrust regulators told them so. This clearly shows how rotten EU antitrust committee is and how easy it is for a well settled lobby to force the whole EU to accept absolutely stupid and nonsense act.

Who will gain any profit from this?

Users? No, they will have to find a way to get any browser on their new computer. And what if it’s their first computer? How are they supposed to download anything from the internet? Yes it’s a five minutes job for an IT specialist, but go teach your grandmother how to use ftp.exe to download anything ;)

Hardware vendors? No, they were always able to install any browser on the computers they sell and make it the default one.

Other browser vendors? To a degree. Anyway Microsoft will provide hardware resellers with an IE8 pack – so they can install IE8 on all the computers they build.

Is there any sense at all in this decision? I can’t see it.

I just don’t get it, really. If I’m buying a car, I won’t argue that the tyres on this new car are from Pirelli (or any other vendor). I will just change them if I don’t really like Pirelli, or choose another car which doesn’t have Pirelly. I’m not prohibited to change them. I’m just given the default one.

And now compare this situation with Apple who clearly prohibits installing Opera on its iPhone. Because it would compete (and, obviously, win the competition) with their Safari browser. Why doesn’t EU antitrust committee look at this? Why don’t Opera take any action here?

Share :