Archive

Author Archive

Improving the DATA act – open govt in the USA

July 8th, 2011

Beth Noveck, NYU law professor and previously Deputy Chief Technology Officer of the US, and I have written a  blog posting that discusses the DATA act, a proposed US law to improve the transparency of US spending. I realized this may also be of interest to folks working on Linked Data, and especially linked open government data, so I thought I’d mention this here as well.

VN:F [1.9.22_1171]
Rating: 10.0/10 (1 vote cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Unanticipated consequences: Saving data.gov

April 14th, 2011

I had a bizarre dream last night, one of those surreal shockers. The details aren’t important, but I realized on waking up that the dream’s theme was all about unanticipated consequences.  I realized I needed to write this post.

To set some context: I went to bed upset last night.  I was upset at two things, one is an article on techcrunch entitled “Five Open Questions For Data.gov Before We #SaveTheData,” the other was my response to the article.  I hope I can respond to the first and apologize for the second.  I want to make one thing clear, however, before I start – I am a strong supporter of http://data.gov, I think it is a great experiment in democracy resulting from bold leadership, and if it dies in the current budget cutting it will be an enduring embarrassment for the USA and a major loss to government transparency.

The article I was upset about was written by Kate Ray (@kraykray), an amazingly bright and articulate young woman who has made several very impressive videos and online articles that I am a fan of.  She recently was one of the co-founders of “NerdCollider,” a website designed to bring intelligent discussion to interesting issues — an idea I support.  I was proud to be an early contributor to one of their discussions, which asked “What would you change about Data.gov to get more people to care?

In the TechCrunch blog post I mentioned above, Kate takes several quotes from this discussion and reflects on their import — is data.gov taking some of the key issues into account?  As a good reporter, Kate’s OpEd is actually quite objective – she reports on several comments made by people, including me, as to issues the site has in terms of its effort to share government data.   TechCrunch is a very influential site, the article title has been tweeted and retweeted hundreds of times to hundreds of thousands of potential readers (congrats to Kate on this viral takeup), raising awareness of Congress’ narrow-minded goal of killing the project, which I guess is a good thing.  Unfortunately, the choice of the word “Before” in “… Before we #savethedata” has a negative implication, and I’m hoping that doesn’t kill off the positive efforts that the #savethedata meme was designed to promote.

In her article, Kate brings up important issues, but what she doesn’t make clear is that most of the people she quotes are indeed strong supporters of the Open Government movement and fans of Data.gov.  The seeming criticisms were actually constructive responses to the question of how we could get more people to care (a positive), and not meant to say what was wrong with the site that must be fixed before the site was useful.  It’s already very useful, but like any new effort, there’s always room for improvement. However, those changes will never happen if the site is forced to go dark!

As I said, Kate’s article has been phenomenally well tweeted, in fact, if you look at #savethedata the stream is so filled with pointers to this article that one can no longer easily find the link to the Petition created by the Sunlight Foundation to help stop the budget cuts — that petition is where the #savethedata meme started (thanks @EllnMiller).  Kate also doesn’t point to the great HuffPost article by @bethnoveck explaining why cutting the funding to this and other egovernment sites will threaten American jobs which was also retweeting around the #savethedata meme.

So I hope one unanticipated consequence of this article is that it doesn’t help cause the death of data.gov by killing off the awareness of its importance or losing the momentum on the petition that could save it.

But, as Arlo Guthrie used to say, “that’s not what I came here to talk about tonight…”

In my response to Kate’s article, I referred to her making factual errors.  This is a horrible thing to accuse a young journalist of, and I was being unfair.  The errors I wanted to point out were not in Kate’s piece, but in the chart chosen to go along.  It appears to show a flatline in the interest in data.gov, using figures from (as Kate told me later in a separate tweet) compete.com on “unique visits.”  I don’t know where compete.com gets the data, but the tracking of the  number of visitors on the data.gov site — which are reported on the site on a daily basis seems to show a much larger number with a more positive trend (over 180,000 visits in March).  It’s unclear why there is this discrepancy (I suspect it’s in how compete.com figures uniqueness for sites they don’t control), but it is clear it isn’t Kate’s fault.   She also cites the number of downloads in her article as 1.5M since Oct 2010, which is the number reported on data.gov, but as of last week, the site broke 2M downloads, and the number is trending up.

Anyway, I’m digressing again (occupational hazard of a college professor) — the key point is the errors are not Kate’s and that she was reflecting on what she found.

I also was upset that she quoted me out of context – in my nerdcollider response I made it clear I was supporting data.gov, and offering some constructive solutions to the question of how we could make the site better.  As the quote appears in her piece, it looks like I’m saying the data is poorly organized on the site — but what I was actually saying is that in the incredible richness of  data sets available (data.gov hosted over 300,000 datasets at last count!) we have to explore new ways to search for data  — it’s a wonderful problem to have!  But I did say what she quoted, and as she pointed out to me, correctly, one of the good things about nerdcollider is that the full context of the quotes are there to be cited.  She’s right.

So just as I hope Kate’s piece doesn’t have the unanticipated consequence of hurting data.gov, I hope my admittedly intemperate response doesn’t have the unanticipated consequence of hurting the reputation of this young potential online media star.

@kraykray – I apologize.

VN:F [1.9.22_1171]
Rating: 10.0/10 (1 vote cast)
VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)
Author: Categories: linked data, open data, personal ramblings, twitter Tags:

Data.gov – it’s useful, but also could be better.

April 5th, 2011

The “Nerd Collider” Web site invited me to be a “power nerd” and respond to the question “What would you change about Data.gov to get more people to care?”  The whole discussion including my response can be found here.  However, I hope people won’t mind my reprinting my response here, as the TWC blog gets aggregated to some important Linked Data/Semantic Web sites.

My response:

I was puzzling over how I wanted to respond until I saw the blog in the Guardian – http://www.guardian.co.uk/news/datablog/2011/apr/05/data-gov-crisis-obama – which also reflects this flat line as a failure, and poses, by contrast, the number of hits the Guardian.com website gets. This is such a massive apples vs. oranges error that I figure I should start there.

So, primarily, let’s think about what visits to a web page are about — for the Guardian, they are lots of people coming to read the different articles each day. However, for data.gov, there isn’t lot of repeat traffic – the data feeds are updated on a relatively slow basis, and once you’ve downloaded some, you don’t have to go back for weeks or months until the next update. Further, for some of the rapidly changing data, like the earthquake data, there are RSS feeds so once setup, one doesn’t return to the site. So my question is, are we looking at the right number?

In fact, the answer is no — if you want to see the real use of data.gov, take a look at the chart at http://www.data.gov/metric/visitorstats/monthlyredirecttrend — the number of total downloads of dataset since 2009 is well over 1,000,000 and in February of this year (the most recent data available) there were over 100,000 downloads — so the 10k number appears to be tracking the wrong thing – the data is being downloaded and that implies it is being used!!

Could we do better? Yes, very much so. Here’s things I’m interested in seeing (and working with the data.gov team to make available)

1 – Searching for data on the site is tough — keyword search is not a good way to look for data (for lots of reasons) and thus we need better ways – doing this really well is a research task I’ve got some PhD students working on, but doing better than is there requires some better metadata and approach. There is already work afoot at data.gov (assuming funding continues) to improve this significantly.

2 – Tools for using the data, and particularly for mashing it up, need to be more easily used and more widely available. My group makes a lot of info and tools available at http://logd.tw.rpi.edu – but a lot more is needed. This is where the developer community could really help.

3 – Tools to support community efforts (see the comment by Danielle Gould to this effect) are crucial – she says it better than I can so go read that.

4- there are efforts by data.gov to create communities – these are hard to get going, but could be a great value in the long run. I suggest people look to these at the data.gov communities site, and think about how they could be improved to bring more use – I know the data.gov leadership team would love to get some good comments about that.

5 – We need to find ways to turn the data release into a “conversation” between government and users. I have discussed this with Vivek Kundra numerous times and he is a strong proponent (and we have thought about writing a paper on the subject if time ever allows). The British data.gov.uk site has some interesting ideas along this line, based on open streetmap and similar projects, but I think one could do better. This is the real opportunity for “government 2.0” – a chance for citizens to comment just on legislation, but to help make sure the data that informs the policy decisions is the best it can be.

So, to summarize, there are things we can do to improve things, many of which are getting done. However, the numbers in the graph above are misleading, and don’t really reflect the true usage of data.gov per se, let alone the other sites and sites like the LOGD site I mention above which are powered by data.gov.

VN:F [1.9.22_1171]
Rating: 10.0/10 (1 vote cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Budget Cuts Threatening Data.gov

March 31st, 2011

You may have heard that the Data.gov website is going to be shut down.  I wish I could say this is completely false, but  I can at least say that it is a bit premature — if Congress cuts the budgets to the threatened level, a number of sites, including Data.gov will have trouble continuing to grow, and some may have to be shut down — but right now the budget cuts are not final, and the plans are still in the works.  Data.gov, luckily, is less expensive than some of the other sites to maintain, so the discussion right now is more about cutting plans for expansion than shutting down completely, but even that would be a major blow to open government data. However, sites like USAspending and others will be harder to maintain, and even data.gov could end up shut down if the full cuts go through unchanged (but at least I’m personally hoping the Senate and White House will resist this)

What you can do is to get involved!  Let your politicians hear from you — the Sunlight Foundation has a great site about this at http://sunlightfoundation.com/savethedata/ which will let you sign a petition and has some suggestions for other actions.  It also has up to date information on the situation — please go look there.

There’s also a lot of articles out there, and much to follow in twitter space — here’s some starting points

In the past day, there have been a lot of articles in the news about Data.gov:  http://www.google.com/search?q=%22data+gov%22&hl=en&prmdo=1&tbm=mbl&num=10&lr=&ft=i&cr=&safe=images&tbs=qdr:w#q=data.gov&hl=en&lr=&prmdo=1&tbm=nws&ei=ehuVTfz9GY3msQOCiJ3MBQ&start=0&sa=N&bav=on.2,or.r_gc.r_pw.&fp=83f1e1e6450f219c

A good article by Beth Noveck (I’m the president of her fan club :-) ): Huffington Post: “Why Cutting E-Gov Funding Threatens American Jobs
http://www.huffingtonpost.com/beth-simone-noveck/why-cutting-egov-funding-_b_840430.html

The hashtag for following this on twitter is #savethedata

So please, join us in saving these important government transparency efforts!!

-Jim Hendler

p.s. For some irony, Hong Kong’s open data site went live today: http://www.gov.hk/en/theme/psi/welcome/

Here’s some more articles and things for those interested

Federal News Radio, Daniel Shuman, Sunlight Foundation, “Budget cuts may end transparency programs”  http://www.federalnewsradio.com/index.php?nid=17&sid=232614
Federal News Radio, Executive Editor, Jason Miller, “OMB prepares for open gov sites to go dark in May”: http://www.federalnewsradio.com/?nid=35&sid=2327798
Sunlight Foundation, Daniel Shuman, “Budget Technopocalypse Deepens: Transparency Sites will go dark in a few months”: http://sunlightfoundation.com/blog/2011/03/31/budget-technopocalypse-deepens-transparency-sites-will-go-dark-in-a-few-months/
Washington Examiner, Mark Tapscott, “Transparency advocates appeal to Congress to avoid budget cuts”: http://washingtonexaminer.com/blogs/beltway-confidential/2011/03/transparent-advocates-appeal-congress-avoid-budget-cuts
PCWorld, Grant Gross, “Group Protests Proposed Cuts to e-Government Transparency Efforts”: http://www.pcworld.com/businesscenter/article/223618/group_protests_proposed_cuts_in_egovt_transparency_efforts.html
“Data.gov and 7 other sites to shut down after budget cuts”: http://www.readwriteweb.com/archives/datagov_7_other_sites_to_shut_down_after_budgets_c.php

VN:F [1.9.22_1171]
Rating: 9.3/10 (3 votes cast)
VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)

Semantic Web Resolutions: 2011

January 3rd, 2011

Have attempted to elucidate at least part of my research agenda for the coming year — blogged at: http://blogs.nature.com/jhendler/2011/01/03/semantic-web-new-years-resolutions

-Jim Hendler

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)
Author: Categories: tetherless world Tags: