/Dan/.NET - Dan Robey's .NET Dumping Spot : /maniacal

CreateDebate

dsrobey — Thu, 24 Apr 2008 22:51:00 GMT

I've been helping some friends with a website recently and it's just entered a public beta. CreateDebate.com is an online debating community that lets users make new open debates and see what people think about issues. Check it out, it's easy to create new debates and arguments and check out where everyone stands.

XAML layers

dsrobey — Mon, 26 Apr 2004 16:26:00 GMT

What have people heard about the new Avalon / XAML programming practice MS seems to be pushing? To me it seems almost parallel to the Java/Javascript division, with new twists thrown in. What it promises (if it is integrated into IE as I think it will be) is a programming environment in which rich media applications similar to flash can be thrown onto a website and be dynamically loaded into a sandboxed environment on the client side. If we think about the push to portal systems and the internet as storage space, this is a big step in that direction. Pieces of software can be loaded and unloaded, taking advantage of web services to provide local/networked actions. Think of a networked MATLAB where you interact through a local presentation layer while all the number crunching is done by server farms. In theory, you could make an entire networked operating system using this stuff

Dividing presentation and implementation is something we've been trying to do for a long time, but this seems to even more strongly enforce it. Talk about Macromedia providing a development environment for the presentation layer would make the top layer even slicker, while the implementation is done through "partial classes." This means not having to learn the strange flash hooks for web services but still being able to integrate rich displays with solid framework.

dynamic pictures

dsrobey — Sat, 24 Apr 2004 23:14:00 GMT

I was thinking about how to get around the limitations some sites have on the http-referer for pictures (mostly webcomics that don't want deep linking of pics), and I came up with an interesting idea. Using the strange qualities of Response.Write() and Response.End(), I made a page which takes as input which picture it should get (does a n-1 tier lookup for the http-referer, other info), and then downloads that picture and serves it to the client.

Confused? You should be. Since I whipped it up in about ten minutes, the code's not the cleanest, but it'll give the basic idea:

if("pic".Equals(Request.Params["pic"])) { System.Net.WebClient wc = new System.Net.WebClient(); wc.Headers.Add("Referer","http://rx7.mit.edu"); string newimage = this.Server.MapPath(".")+"\\pics\\"+System.Guid.NewGuid().ToString(); wc.DownloadFile("http://rx7.mit.edu/help.gif", newimage); Response.WriteFile(newimage); Response.End(); System.IO.File.Delete(newimage); }

So the image reference looks like <img src=“$page?pic=pic“>, where $page is your web app's address.
Instead of downloading the picture from somewhere, it could be dynamically created, using all of .NET's image manipulation libraries, say, to show the time, or some graphic representation of any data .net handles. Or do checking before serving an image to see if the user has correct permissions. Anything your heart desires.

dealing with long queries

dsrobey — Thu, 04 Dec 2003 19:13:00 GMT

One of the problems with long web queries is user feedback. If a SQL command takes seconds or even minutes to complete, or if some other heavy processing must be done, users must wait idly as the status bar creeps along, blindly guessing at how long the response will take.

This problem is a general notification problem with web queries which stem from the nature of stateless http client/server protocol. A possible solution is to place a layer of indirection on anticipated long queries. Using the web service model, the indirection layer (call it the foyer layer) does the standard query asynchronously, and instead of sending the results, sends a java refresh page, which causes the client to requery the foyer layer every n seconds. The foyer layer can hold data about the current query, including what percent is done. This can be relayed from individual calls, if using SQL can be updated after each SQL statement in the long SQL stored procedure, or previous long calls can be used as an estimation metric. The data can be used to update a true status bar on the page while serving other information to the client - funny quotes, relevant advertising, customer feedback, etc.

The foyer layer can be programmed in any language, and one method which comes to mind would be the recent Shockwave web services, which allow for a stateful client.

sql injection attacks

dsrobey — Thu, 20 Nov 2003 19:02:00 GMT

While your head is still spinning from the last post, don't think that SQL server is the answer either. If an adversary knows your SQL server setup, they can find weaknesses in your database calls and do "sql server injection attacks." Basically, since most SQL programmers assume the strings are nicely encoded already, you can send sql commands as your string, which when using function calls like
"SELECT blah FROM blah WHERE blah=$input"
the attacker can send back inputs like
"' ; INSERT INTO adminusers 'Dan Robey'"
If crafted properly (my strings are just examples and won't work), these inputs will end the current query and instead do the "evil" query. This is one reason why everyone using SQL should disable that convenient little "xp_cmdshell" system call in their SQL server. Otherwise some hacker can just send "' ; xp_cmdshell("format c:")" or something equally nasty.

Hope after reading this you all feel safe and secure with the way your system is setup!

security issues with xmlportal

dsrobey — Thu, 20 Nov 2003 18:51:00 GMT

I just realized that since our xml data files have to be set ASPNETuser readable, if an adversary knew their location, he could read all the data, including users and passwords. I'm not sure how much is needed, but here are some possible solutions depending on the security desired:

If we want to protect all the information, we can symmetrically encrypt the file with some machine dependent salt plus extra passphrase. On the first serialization call we need to decrypt, which might take about half a second. Weakness: if the salt algorithm is known, and machine dependent secrets are known, the key can be found by looking through the code.
If we are just worried about the user passwords, we can take the standard approach of storing not the password but the SHA1 of the password plus some salt. More secure, but has dictionary attacks.
In addition, we can give the XML file permissions such that only users requesting from the local machine can get it. This makes data theft harder, since adversaries would have to spoof their origin IP addresses, especially hard if the server is using NAT.

I'm going to have to think about the problem a little more before coming up with a good way of doing this. I'll definitely implement the SHA1 though.

Some decent references I came across were this msdn article on "building ASP.NET applications" securely, which gives good code examples, and another on protecting sensitive strings in .NET code in general. Of course, in the end what they rush over quickly is that if you are storing sensitive keys (for SQL calls) in code, there will virtually always be a way to get to it, through clever reverse engineering, so if at all possible, don't do it!

dual layer dvds

dsrobey — Sun, 19 Oct 2003 05:00:00 GMT

just noticed today that you can actually tell if a dvd is dual layer just by looking at it. The grooves in the two layers create interference and create a kind of moire effect. Single layer dvds don't have the interference and they have a flatly mirrored surface.

re: using blogs as database storage

dsrobey — Tue, 07 Oct 2003 20:58:00 GMT

In response to the feedback from my last blog agent post, the limiting factor in the blog agent project is definitely not the bandwidth. By no means am I attempting to engineer a Google-scale application, searching billions of pages. This is a much smaller spider that only hits the specific sites I (and other people signed up for the service) want to watch. Now, if the number of pages to filter became in the millions, that might be a little much, even for the T3 line the site is hosted on.

For a very simple example of what I'm talking about, check the rss feeds page of the site, which currently grabs rss feeds hourly, caching them on my site. Some of the rss feeds shown on the page are scraped directly from the site, either by myself, in the case of the two deal sites, or by others, in the nytimes feed. In the final version, there would also be a "blog agent" alert feed which would show the filter results.

The nice thing about what I like to call "upstream filtering" is that sources do not have to be the raw html pages. For example, there are already communities who sort through amazon.com, dell.com, bestbuy.com and post their results online. The output of these communities can be filtered instead of doing scrapes of amazon.com, etc. Examples of upstream news filtering already in operation are fark.com and slashdot, among many others.

The difference with my system would be that it implements filtering triggers which automatically create the content. This is where things become more difficult. Well crafted keyword filters are enough on many sites, although in some, it may be necessary to resort to regular expression parsing, which although I'm not a big fan of, definitely is a powerful tool. In part, this section is already dealt with by the human filters that post stories to news aggregation sites, eliminating inane or repeat stories (as I see it) so that my agent has to do less work.

using blogs as database storage

dsrobey — Mon, 06 Oct 2003 17:12:00 GMT

The real reason I wanted to have a blog web service, as I stated in my last post, is that I'm looking at storing web data I scrape off the internet in a blog or blog-structured database, an idea I disucussed a while back . The nice thing about the blog data structure is that it contains many of the fields that one would want if they were designing a database for web scraping anyway, with categories, extra links, time posted, etc.

The blog web service would be used by the spider to populate these blogs. Whenever it found a link that resembled "cheap mp3 players," or whatever was defined as the search params, it would call the blog web service, send it the data structure wrapped in SOAP headers, to the "add entry" function of the web service, and be done. Ideally the blog web service would have all user functionality a normal blog server would have, including adding new blogs, editing entries, deleting entries, categories, all with a nice WSDL file describing them for programming ease. However, for the moment all I'm looking for is the "add entry" function.

The nice thing is that the data stored in the blogs would look just like a normal blog to a rss reader or blog web control, and it would display just as nicely. Once the blog web service is written, it becomes trivial to implement the upstream blog filtering project I described.

more on wearable computing

dsrobey — Thu, 11 Sep 2003 17:58:00 GMT

browsing around for more info:

weblog mentioned on a slashdot about the BBC article with a little more info.
the whitepaper on MIT group's UI. They bring up some vaild points about the display being very small, task management in that environment, etc. I still think it needs a nice skinning.
Apparently last IAP they integrated the Sharp Zaurus into their design for ease of programming. Very nice.

They also describe nicelgy some of the reasons and benefits behind wearable computing, and their hardware construction. This is all good for technical details, but the end user doesn't care about that. They care about being able to check their email by pressing a button, or having voice-over-IP seamlessly. MIThril is far from that.

Oh, and also info about a shoe-based power supply that provides power from walking.

wearable computing

dsrobey — Thu, 11 Sep 2003 17:21:00 GMT

A recent BBC article highlighted the oncoming wave of wearable computing devices. While currently the iPod is the biggest hit, with any luck pervasive computing will grow. Some issues still need to be resolved before this becomes a reality:

A power supply currently can only run a laptop for about 3 hours. While wearable computers will hopefully need much less power than that, a light, rechargable, high power battery solution needs to be found for the always-on wearable devices. Perhaps something that created energy from compression while walking or solar arrays
Interface design. Current software has desktop oriented design, which does not translate to wearable design well at all. The MIT wearable computing center uses a crufty windows manager, which will not translate well to slick applications. Perhaps some form of wsdl for apps needs to be created.
another solution to interface design would be a well-written audio I/O. If a totally verbal "windows manager" could be created, this would solve all interface problems. This presents many hurdles, but the end would be worth it. Microsoft has a S/T interfacewhich might be helpful. Heres's an intriguing sample of ease of use of the SDK.
killer app: something really really cool that appeals to everyone. MP3 players abound. For a price wireless enabled cell phones can be had. Palm pilots are a step in the right direction, but a series of games perhaps, which use the wearable devices. Something like real-world quake arena, where troops can prowl the streets and their "glasses" overlay a halo around enemy team members. Kind of like laser tag, but always on.

blog subscription

dsrobey — Sat, 06 Sep 2003 20:25:00 GMT

bored, so I'm going to try and think through the blog subscription problem. The system should have 3 parts: the frontend, which users sign up for blogs in, including the keyword filter section, the backend, which uses the subscription information to locally cache rss and filter through the files, and a middle component, which would do any messenging or dynamic feed creation.

Let me clarify the design params. I want users to be able to sign up for different rss feeds which are shown when they hit a webpage. Once this functionality is in place, I want to be able to do rss filtering, including keyword filters, to create dynamic "personalized" feeds. Once this is working, Users should be able to have any items matching their filters immediately sent to them through IM, email, etc.

My next few posts will deal with issues surrounding this problem.

blog ratings

dsrobey — Wed, 03 Sep 2003 18:00:00 GMT

Andy has a good rating idea for his blog rating system. However, blog syndication is hard to track, since not all people use the standard methods of syndication. A better idea might be to use a google-like system of, where the rating is determined by how many links there are to a certain blog.

Google would be the obvious implementer of this system, since they've already worked out a lot of the bugs in such a rating system. However, a simple implementation would be to get all the rss syndication fields and to scan through them for links to other blogs. Each link to a blog would count as a point (no normalizations as in google), and the total point count would be the rating of the blog. In a good blog search engine, this rating would be combined with the word frequency to determine relevance.

A more complicated system would use a more google-like implementation with the rating determining link strength, algorithms to prevent bogus link pages from boosting one's ratings.

alternative web services

dsrobey — Thu, 28 Aug 2003 23:08:00 GMT

Today I heard about an interesting way to check the weather. The weather network has an image at http://www.theweathernetwork.com/inter/weathercentre/weatherbutton/images/usa/BOS.gif, which updates every so often to show the current weather status in boston. This had me thinking about how many alternatives to web services I see daily. Most of these have interesting ways of transferring the data along the wire. The blog server uses an ISAPI filter to find the correct user instead of having using a more simple, but slightly less pretty request header, as in having http://blogs.mit.edu/?dsrobey redirect to my blog. So the query information is actually in the form of a malformed http request which the server spots. Another example of this is the page counter at the bottom of many web pages. The query for the graphic has a GUI in it which maps it to the user's statistics as the server serves the image. Can you think of other examples of interesting inputs to generate web-service like outputs?

ramifications of rss filterer

dsrobey — Tue, 26 Aug 2003 06:03:00 GMT

the rss searcher could also be used as an efficient form of upstream blog filtering, as discussed briefly in . To reiterate, if a group wanted to have a site dedicated to mentions of longhorn rumors, or SCO rants, they could syndicate many news sources which already provide rss feeds and filter out the ones not relevant to their site. The rss search robot could systematize this for them. If results were stored in another feed, they could just plug in the search string "SCO" and they're basically done.