Havoc's Blog

Just hooked my DAV implementation up to real shared files in
the Mugshot shared
files feature I mentioned last week.

Unfortunately, it looks like this won’t work all that well with
the DAV implementations in Nautilus or Windows Explorer.
Both ignore the “display name” provided by DAV, and neither have a
way to provide an icon URL via DAV. They do a fine job of browsing the
DAV share but the filenames and icons are all fugly.

For Nautilus I think the only real solution is to provide a tree of
.desktop files instead of a tree of real files (ugh). For Windows, I
think there are third-party DAV implementations that don’t suck as
much. For either one another option is to write a custom vfs backend /
shell extension, though for Nautilus even a vfs backend requires the
“virtual .desktop files” hack to override display name and icon.

btw, I think DAV would be a nice model for an API replacing
gnome-vfs. The main thing DAV does is add a method called PROPFIND to
http. PROPFIND is sort of a combined recursive readdir/stat.
It lets you recursively list a directory, simultaneously gathering any
properties of interest, in a single round trip.
Properties are identified by a namespaced XML element name, so there’s
a simple extension mechanism for adding new ones.

The other http extensions in DAV are simpler than PROPFIND and add
the ability to create directories and modify properties. There’s also
an optional file locking protocol.

A vfs library based on this would have core operations of GET/PUT
entire file, and PROPFIND to list directories. On top of the standard
DAV properties, an icon property would be an obvious addition, and
also properties for the UNIX-specific stat() information. Backends
would return only properties that made sense for that backend.

VFS backends would be MUCH easier to write if they only had
to support GET, PUT, and PROPFIND. They would also be much easier to
write if they could provide display name and icon properties
instead of making up fake .desktop files.

Operations such as seek, etc. would be optional and implemented only
for a local file backend.

Another way to simplify backends is to do path-parsing in the vfs
library itself instead of in the backend, I did this for the Mugshot
DAV implementation,
here
is the API for (read-only) backends. Every time I’ve implemented
path parsing (gconf, dbus, Mugshot DAV, probably elsewhere) I’ve found it hard to
get right and tedious, so it would be nice to remove this burden from
backends. The only trick with a node-based backend API is to be sure
there’s a getChild() in addition to getChildren() to avoid gratuitous
directory listing / node creation just to resolve a path.

A possible incremental approach to replacing gnome-vfs on the DAV
model could initially have a very small library (with just the PUT,
GET, PROPFIND operations) that defined the backend API. To start,
there’s a gnome-vfs backend that calls this library. gnome-vfs
continues to handle the volume manager and stuff like that. But
backends could immediately be written on the DAV model, and gnome-vfs
backends could be incrementally moved to it.

It might be interesting to map even more literally to DAV and have
requests to backends look like http requests, with a method and
headers. This would make the backend API extensible over time by
adding new methods. Adding new file properties is probably more
common, but new methods are useful for e.g. adding a seek operation
that only works on local files, or whatever.

The value of DAV here isn’t that it’s a standard (we’re talking about
a C API that looks like the DAV protocol, not an implementation of
the protocol itself). The value is rather that it is roughly correct (few
operations, designed to be asynchronous, extensible), and by using an
existing model one avoids a lot of design decisions and discussion.
Designing a new VFS API is really the hard part, so if that can just
be punted to DAV then (assuming it works) it could save a lot of time
and energy.

A side benefit is that it’s easy to interoperate with remote DAV
shares, either exporting a vfs share as DAV or mounting a DAV share
with the vfs, and there’s no loss of information or semantic mismatch
when doing this.

(This post was originally found at http://log.ometer.com/2006–09.html#9)

Coded most of a file
sharing feature for Nautilus over the holiday weekend, reposting
here so it will show on Planet GNOME.

BTW, AbiWord
Collaboration looks awesome. Once the site launches I’d like to
investigate how it can be Mugshot-integrated so I can get a stacker
block when someone I know starts a public document.

(This post was originally found at http://log.ometer.com/2006–09.html#5)

Hmm, Luis
mentions this AOL-as-badware
report. While some of the stuff is plainly bad, such as the
fact that you can’t uninstall fully, I can’t agree with some of
it.

e.g. Google Talk also does automatic updating, and I think it’s
exemplary good software behavior; the alternative is all the apps that
install a “check for updates” tray icon and tell you about it all the
time. “Look at me! I’m updating too! Check it out!” — cute for one
app, sucky when you have quite a few installed. Hmm, Firefox also
does automatic updating; it auto-downloads, then asks if
you want to restart Firefox, and whether you do or not the next time
you start up Firefox you automatically get the new version. Are these
badware too?

I don’t think putting a toggle for this in an installer would make
any sense. Many people wouldn’t know what it meant or wouldn’t read
carefully, and if they turned it off they’d just be shooting
themselves in the foot by not getting security updates. Why clutter
the installer? Allowing auto-updates to be turned off in preferences
is just fine.

Seems like some of these “badware” guidelines lead to “dialogware”
(which is also a nasty Windows disease), where instead of just picking
sane behavior and allowing you to modify it later, the app asks you
about everything in a dialog. As I think Joel notes, programmers love
dialogs, but in my experience designers will LART you pretty much every
time you put one in. I also suspect that
technical users are more prone to want to be asked everything than
nontechnical users are. (Random aside, Visual Studio is the worst
in this respect, it’s like a dialog firehose.)

Anyway, I’m not sure the badware guidelines should include
debatable design decisions or stuff that’s merely annoying. As long as
the app uninstalls completely and easily from the normal add/remove
programs screen, if people think the app sucks they can always get rid
of it. Of course, the AOL software does appear to be hard to get rid
of.

I have to admit I was surprised and annoyed when the AOL Instant Messenger
installer also installed some crazy-weird web browser (not IE, not
Firefox) … the bundled software with AOL 9.0 could conceivably be
there as dependencies (e.g. maybe AOL 9.0 requires Quicktime to work
right), but it’s tough to imagine that an IM client needs to replace
my web browser in order to work.

(This post was originally found at http://log.ometer.com/2006–08.html#29)

Deadly trap all Java programmers should know: URL.equals() and
URL.hashCode() resolve the hostname in the URL, then compare equality
by IP address. Which means 1) these methods are way too slow and 2)
virtual hosts break them — every RSS feed URL on blogspot.com (for
example) compares equal.

This innocent-looking
code to cache RSS feeds in ROME is thus totally busted.

(This post was originally found at http://log.ometer.com/2006–08.html#12)

Two months ago we launched a limited user trial of our Mugshot
prototype, and have learned a lot since then, both about the user
experience and about how to keep the server from falling over. Thanks
to everyone who’s been helping out, we’ve had good comments, advice,
and patches.

On the user experience front we’ve decided to pull together a
number of new features we’ve tried over the last couple months into a new design we’re code-naming the
“stacker”. This pulls some of the flavor of where I’m at into the
primary Mugshot UI, letting you easily keep track of everything that’s
happening online in your network of friends and interests.

I’m implementing a web-based prototype of the eventual Linux/Windows
client interface right now. You can see the javascript
code here
and server side code here (sorry websvn is so slow).

The goal is to put up an ugly, but usable, web-based prototype
probably sometime next week for people to try. Once we get the basic
interactions about right (and hopefully verify that this direction
makes sense), we’ll do a more web-friendly design and replace
the main Mugshot page with that. We’ll also code the Windows and Linux
desktop versions.

In parallel, we’re investigating server scalability and will be
redoing things as required to spread the load over two servers
(right now we still have only one). The big step is from one to two of
course, from two to four is in principle much simpler. Ideally this
will come together about when we have the new UI.

(Another shorter-term task, Owen is finishing up the improved Firefox support.)

If it all pans out we should have both a better user experience and the
ability to scale up as we add more users — woot.

One effect of the new direction is that Mugshot could
offer a very interesting API, both via web services and over D‑Bus/COM to
desktop applications. This API could let you get people’s profile
information including pointers to their stuff on Flickr, YouTube,
blog; list of friends; email addresses; etc. (getting some of this
would require you to be logged in as a friend of the person).
It could also allow apps to get live notifications when things
happen, such as new Flickr photo sets, new links shared, new friends
appearing, new comments on your blog, new songs played, or whatever it is.

We have to focus on our two current goals of implementing the
“stacker” and server scalability basics. This keeps us from fleshing out the API
too much right away, but if there are specific API calls you’d like to
have or experiment with in an app, please let us know.
The other night I added a really lame API
docs page using introspection (sometime next week you can probably see it on the live site, right
now it’s only in Subversion).

(This post was originally found at http://log.ometer.com/2006–08.html#11)

I’ve been known to complain about
Javascript before. The thing that’s bugging me this week:
attempts to claim its misfeatures are a positive.

First we have this
tutorial on splice():

Using the example, try entering bad data in the
boxes. Try entering letters (numbers are needed). Try entering numbers
too large or small for the array. You’ll notice that nothing
happens. The program does not crash even though I’ve coded no error
control.
One of the best features of using array methods to update dynamic
arrays is that errors are automatically handled with a default action
of nothing.

Wooohooo! My program silently does nothing when it has a huge bug in
it!

OK, that may be debatable. But how about
this
defense of the Javascript class/inheritance “system”:

JavaScript can be used like a classical language, but it also has a
level of expressiveness which is quite unique. We have looked at
Classical Inheritance, Swiss Inheritance, Parasitic Inheritance, Class
Augmentation, and Object Augmentation. This large set of code reuse
patterns comes from a language which is considered smaller and simpler
than Java.

Gee, I live to choose among 5 kinds of inheritance, all of them
inconvenient and bizarre.

Unlike sane dynamic languages such as Python, Javascript isn’t even
typesafe at runtime… if you pass the wrong number of args that’s
cool, if you pass the wrong kind of object that’s cool too, assign to
a nonexistent field — let’s just create that!, use a string instead
of an integer — let’s just convert it! — do inheritance wrong — oh,
you’re inventing a cool new kind of inheritance! Sweet!

The hype about Ajax has to be amazement that anyone got this
crappy language to do anything at all.
Google has the right
idea:

Google Web Toolkit (GWT) is a Java software development framework that
makes writing AJAX applications like Google Maps and Gmail easy for
developers who don’t speak browser quirks as a second
language. Writing dynamic web applications today is a tedious and
error-prone process; you spend 90% of your time working around subtle
incompatibilities between web browsers and platforms, and JavaScript’s
lack of modularity makes sharing, testing, and reusing AJAX components
difficult and fragile.

That about sums it up…

If you think I’m overreacting you probably haven’t written and
maintained enough Javascript 😉 and we would be
very happy to hire
you so you can get more practice. You think I’m kidding, but I’m not…

(This post was originally found at http://log.ometer.com/2006–08.html#10)

A team at Red Hat is looking for people to take this survey
entitled “what do you think of us?” — if you have a chance I’m sure
they’d appreciate your help.

(This post was originally found at http://log.ometer.com/2006–08.html#3)

To me open source vs. proprietary isn’t a good vs. evil question, simply
two different things with different costs and benefits. That said,
open source tilts the benefits toward users
over software companies, all else being equal. That’s why open source
evangelism makes sense.

Since the Internet appeared, a
trend in the software industry has been toward “software as a service”; which
can be enterprise-oriented like Salesforce.com or
consumer-oriented like GMail or MySpace. In addition to web sites,
it can include the cable box and cell phone model of
automatically-managed software on a client device.

I don’t think the open source community has changed its thinking as
radically as this trend requires.

First, our standards for
openness lose much of their relevance in the modern world.
Second, products that include both software and
online services are providing much more compelling user experiences
than software alone can offer; leaving open source software at a disadvantage.

GPLv3 — missing the big picture

The FSF is fighting hard for the “no locked down
devices” provision in GPL version
3. GPLv3 says that you can’t have hardware devices which require
software signed by a proprietary key.

But if I think about Richard Stallman’s
printer story, the “locked
down by a proprietary key” case is not the only case to worry
about.

My cable company doesn’t sell me any hardware at
all; they charge me a monthly fee and they put their hardware in
my living room. Have they even distributed the software to me?
Could I install different software on a leased cable box, even if
the software were GPL?

This isn’t some hypothetical situation; I already have a cable box
DVR I don’t own… and while the FSF keeps using TiVo as an example in
GPLv3 conversations, TiVo’s approach is almost the exception to the rule.

Even if it weren’t, in-home hardware devices are a small part of the picture.
Does it really matter whether hardware I don’t own is in my
living room or in some server room?
Can you replace the software on your favorite web-hosted application?
Or even read the source or submit a patch?

In short GPLv3 prohibits one implementation technique for
“software as a service,” because copyright law happens to make
it easy to do so; but GPLv3 has no impact on several other
far more popular implementation techniques in the same spirit such as leased equipment,
web-hosted services, and thin clients. What’s the point?

Addressing hosted services in licenses

The current GPL
v3 draft does have an optional provision 7b4 that gets at this:

terms that require, if a modified version of the material they cover
is a work intended to interact with users through a computer network,
that those users be able to obtain copies of the Corresponding Source
of the work through the same network session

The Open Software
License includes a provision defining external deployments as distribution
(thus the license’s GPL-like requirement to provide source code kicks in):

5) External Deployment. The term “External Deployment” means the use, distribution, or communication of the Original Work or Derivative Works in any way such that the Original Work or Derivative Works may be used by anyone other than You, whether those works are distributed or communicated to those persons or made available as an application intended for use over a network. As an express condition for the grants of license hereunder, You must treat any External Deployment by You of the Original Work or a Derivative Work as a distribution under section 1(c).

Free passes to web companies

There’s some recognition of the danger, then. But the open source
world isn’t excited about it — let’s take Google as an example.
(Update: Again, Google isn’t the point, just an example.)

Like many tech-savvy type people, I use GMail and GTalk. But while Google supports some open source projects,
if I think “secrecy and proprietary IP used to maximize hype and competitive
advantage,” I think of two companies: Apple and Google.

Google is
not coming down against software patents, and while they are quick
to have open APIs, only a tiny fraction of their coder’s output is open
source.

In spite of their proprietary leanings,
Google gets a mostly free pass from the open source
community. Even
Slashdot comments are positive.

Of course Google does cool stuff and
succeeds in their “don’t be evil” mantra most of the time. I use their
software. But they are not an open source company, and it’s
somewhat odd that people who supposedly care about open source rarely
bring this up.

Do people simply not think of open source principles in connection with web
hosted software?

What to do?

What is the solution? The GPLv3 approach will limit GPLv3
adoption, without addressing the important cases. It’d be
better to drop the “TiVo provision” and be sure the Linux kernel
and other projects widely adopt v3. Then approach
software-as-a-service head-on in a more comprehensive way.

First, we need to recognize that software-as-a-service (whether
the hardware is in your living room or a server room) has huge
advantages for users. I don’t want to buy a whole computer when I only
need a small slice of a server; I don’t want to be a system
administrator just to get email; I appreciate that TiVo systems don’t
get viruses since they won’t run unknown software; and maybe most of
all I love features that involve sharing, collaboration, and socializing.

The open source world has to find a way to provide these
benefits, if it’s going to remain relevant.

Second, there are plenty of aspects of open source that might
apply to hosted or automatically-managed software. A community of
developers and designers; the ability to fork; interoperability with a
range of products vs. “walled gardens” and vendor lock-in.

How does software-as-service change
best licensing practices? Is there an “open service definition”?
How does it change community-building? As a practical matter,
how can open source developers build user experiences that require real infrastructure
such as storage and bandwidth?

There are some great success stories of open services already,
such as Wikipedia.

Issues

Open services pose more questions than answers at this point.

While data becomes part of the
“lock in” of many services, it can’t really be made open if it has any
access controls associated; instead there’s a need for privacy
policies and careful attention to who can access the database.

User-contributed content has gotten
quite a bit of attention from Creative Commons and
others. Content licensing might be at least as important as
software licensing for an online service.

There’s a chance that services will become more commoditized over
time; more generic services like Amazon S3 might allow people to
buy hosting separately from the software. Open source might be able to
speed this up by providing software that runs on such services.

Who knows. But it’s about time to make this issue more prominent. I
wish GPLv3 took it on either more strongly or not at all.

(This post was originally found at http://log.ometer.com/2006–07.html#29)

To save people time, here is some already-known info on D‑Bus performance.

Please
read
this post and a
short followup about the new recursive type
marshaling stuff slowing down those numbers a bit. I think there may
be a couple other relevant mails in the archives, but not finding them
right now.

Be sure the build for performance testing
does not have assertions, verbose mode, etc. enabled.
I’d also avoid caring too much about the DBusGProxy layer as it does a
pile of extra copying and is really convenience-over-performance in
many ways. Not to mention it needs basic sanity work in addition to
performance work.

Historical background: the “D‑Bus is faster” and “CORBA is slow” rumors are
purely originated by fanboys, and have never been either the goal or
the case. ORBit2 is nearly as fast as raw sockets.
By contrast, most design decisions of D‑Bus
(the central bus daemon, validation in a paranoid/secure way of all
data, certain ordering guarantees, etc.) were
traded off in favor of robustness, decreased resource usage,
convenience, etc. rather than speed.

The only potentially significant speed wins in D‑Bus I know of: 1)
D‑Bus makes it pretty easy to be fully async and avoid any round
trips, which can be the key performance issue in some cases, in
particular may matter for application startup; 2) the
central bus daemon makes broadcasts/signals more efficient than the
case where any app that wants to broadcast has to connect directly to all listeners.

I should note that the D‑Bus robustness-over-speed tradeoff mostly has
to do with the libdbus implementation, not the wire protocol. So
someone who disagreed could do a different implementation.

Finally, I’d encourage profiling the desktop and apps, rather than
D‑Bus; I would be surprised if D‑Bus showed up much in app profiles,
unless someone starts shoveling giant quantities of data through
it. If someone is doing that then a structural change would probably
work better than optimizing libdbus marshaling code:
dropping D‑Bus from that app,
using D‑Bus in point-to-point mode instead of through the bus daemon,
using D‑Bus async instead of with round trips, or whatever.
Though anyone is of course welcome to optimize the libdbus marshaling code.

(This post was originally found at http://log.ometer.com/2006–07.html#21)

A couple weeks ago I learned several new things from a talk Is
Media Performance Democracy’s Critical Issue? — the short version
is that newspapers used to aim for 100% circulation, but now aim for
“top 40% of income” circulation — subscribers below the top 40%
apparently increase newspaper expenses but advertisers don’t want to
pay extra to reach them. i.e. most advertisers are selling expensive
stuff. The talk goes into more detail and has
other things to say as well, highly recommend reading it.

In the talk, Tom Stites analyzes an issue of the Boston Globe showing
how most of the articles weren’t relevant unless you had enough
disposable income. Was reminded of it with today’s Globe
magazine cover
story, “The $100 meal may be chic, but it’s not for everyone…
Just $11 gets you Picco’s haddock tacos topped with avocado
vinaigrette.”

Wouldn’t a plate of 3 tacos normally be about $3?

(This post was originally found at http://log.ometer.com/2006–07.html#16)