You are browsing the archive for Rufus Pollock.

Talk at OECD Foresight Forum on Big Data

Rufus Pollock - October 22, 2012 in Events, Talks

Today I was in Paris at the OECD to contribute as a panellist to their foresight forum on Big Data. These were my slides.

Slides from Talk at the Future is Open Conference

Rufus Pollock - October 17, 2012 in Events, Talks

Today I had the opportunity to talk at REFORM’s event Future is Open: Open Government and Data Transparency in a session on Data-driven services: Open data and public service reform. Here are the slides.

Open Data Toolkit – Data Micro Services

Rufus Pollock - October 5, 2012 in Musings, Technical

This is an updated version of my previous diagram on DataHub / Data Micro Services. The updated title reflected a slightly different focus in which integration is somewhat looser, services may be used standalone, and there is greater emphasis on a process than a central Hub (i.e. individual services may be used together or alone without necessarily being part of an overall hub though they also can be seen as part of coherent whole). Keywords:

  • Online ETLV = extract, transform (integate)), load (validate), view/visualize
  • Small Pieces, Loosely Joined
  • Open Data Toolkit
  • Micro (Data) Web Services

Opening Up Local and National Government Open Data Conference – Talk by Rufus Pollock

Rufus Pollock - September 26, 2012 in Events, Talks

Slides from my talk today at the Opening Up project Local and National Government Open Data mini-Conference with the title Open Data and the Open Knowledge Foundation.

Javascript Timeline Libaries and Tools – A Review

Rufus Pollock - September 8, 2012 in Javascript, ReclineJS, Research

Note: a fuller, revised, version of this article was posted on OKFN Labs blog: http://okfnlabs.org/blog/2012/12/04/javascript-timeline-libaries-a-review.html

General comments. Timeline libraries consist of various components:

  • Data loading
    • Date parsing
  • Band (timeline) rendering
  • Showing render info on individual items

For me a timeline visualization library need only be the second of these but most that I’ve come across do both and more.

In fact a major issue in my opinion with most libraries is that they are too under-componentized – they don’t separate cleanly into these different components and end up doing everything.

To take one example, the Verite timeline (in my view is one of the best libraries out there) has a whole bunch of its own custom date parsing built in inside an internal utility library which are hard to override or replace and also has a large chunk of code just for loading from google docs and other data sources. (You can of course somewhat solve this somewhat — as I do in Recline by parsing the dates directly and then submitting in a standardized form). Even if library authors do want to include these things it would be good to do it in a way that allowed for a clean separation so that you could just use the parts you wanted.

Propublic Timeline Setter

  • HTML + JS
    • But Requires a build step (using ruby)
  • Very simple and compact design (nice!)

Verite Timeline

  • Very elegant design
  • Includes much more than Timeline (e.g. sourcing data from google docs etc)
  • Mozilla Public License (was GPL)

Simile Timeline

  • http://www.simile-widgets.org/timeline/
  • The original open-source JS timeline but less regularly update and maintained today: “As of Spring 2012, Exhibit is the only Simile widget seeing active development.” and the timeline control has not been updated since 2009 (see this stackoverflow question for more

Chronoline

  • http://stoicloofah.github.com/chronoline.js/
  • Recently developed and updated
  • MIT licensed

Timeglider

CHAPS Timeline

Using Verite Timeline

Some notes on how to use the VeriteCo timeline software based on experience integrating into Recline Data Explorer.

  • Date parsing: it is reasonably particular at the moment re date support and it’s default is the rather unusual yyyy,mm,dd format. ISO format yyyy-mm-dd is not supported (update: ISO format is now supported by Verite).

Notes on Organizations and Organizational Structure

Rufus Pollock - August 27, 2012 in Organization, Research

Somewhat random notes (accumulating over time – first set from early 2012) on organizations and organizational structure.

Ideas

  • Netflix: “Highly aligned – loosely coupled”
  • Common feature of the more high-tech is high / very competitive compensation
    • Netflix: we pay at the top of the range, we lose anyone who isn’t a star performer
    • Valve: our compensation is very competitive
  • CLOU = Colleague Letter of Understanding
    • Every year, each Morning Star employee negotiates a Colleague Letter of Understanding (CLOU) with the associates who are most affected by his or her work. A CLOU (pronounced “clue”) is, in essence, an operating plan for fulfilling one’s mission. An employee may talk to 10 or more colleagues during the negotiations, with each discussion lasting 20 to 60 minutes. A CLOU can cover as many as 30 activity areas and spells out all the relevant performance metrics. All together, CLOUs delineate roughly 3,000 formal relationships among Morning Star’s full-time employees.

Readings

Javascript Links – August 2012

Rufus Pollock - August 19, 2012 in Javascript, Technical

Frameworks

Dependency Management / AMD

  • requirejs
    • Would like more info on this — always found it a bit painful to use
  • yepnope.js – JS dependency manager/requirer
    • Best I’ve used so far (prefer to require.js though require is somewhat different)

Spending Story: California City Bankruptcies

Rufus Pollock - August 5, 2012 in Research, Spending Stories

This is a holding page for research on California City (and State) finances and bankruptcies. Since the start of the “Great Recession” it appears that city and municipal governments in many US states, and especially California, have faced increasing financial difficulties.

Source google doc

Next steps

  • California state budget
  • Try and acquire balance sheet (reserve plus current P&L) for all cities in California
  • Read up on Stockton

Links

Stockton

Official Chapter 9 Bankruptcy info

http://www.stocktongov.com/government/departments/manager/bankruptcy/default.html

On June 28, 2012, the City of Stockton filed a petition for chapter 9 bankruptcy protection with the United States Bankruptcy Court, Eastern District of California, Sacramento, Case No. 2012-32118. The Stockton City Council approved a Pendency Plan on June 26, 2012. The Pendency Plan is essentially the budget and the financial plan that is followed for the day-to-day operations of the City while in bankruptcy. It identifies what expenditures will be reduced or suspended. The City will continue to pay employees, vendors and service providers. The focus of the City’s plan is the restructuring of above market pay and benefits and unsustainable long term debt.

From A506 application (1st of 3): http://www.stocktongov.com/files/Stockton_RestructuringProposal_01_79pages.pdf

From The “Great Recession” has seriously damaged the City of Stockton’s finances. Record foreclosure levels, plummeting home prices, and high unemployment have cut City revenues, at the same time previous labor contracts, generous retiree medical benefits, and increasing debt obligations were driving up expenses. The City Council has declared two fiscal emergencies and negotiated or imposed labor concessions of $25 million and service reductions of $12 million in FY11-12 alone.

Since FY08-09, the City has been forced through lack of funds to reduce sworn General Fund Police staffing by 25%, Fire staffing by 30% and all other staffing by 43%. Programs and services have been reduced to minimum – or below minimum – levels. Sworn Police staffing per 1,000 residents has dropped from a high of 1.52 per 1,000 residents in 2005 to 1.16 currently, and in the face of a rising local crime rate. While violent crime rates dropped 5.5% nationwide in 2010, they were up in Stockton, which ranked 10th in the U.S. with 13.81 violent crimes per 1,000 residents. [emphasis added]

Despite four years of cost-cutting, current estimates still show a significant shortfall for FY12-13. The City has spent its reserves and cannot close the gap without further large cost reductions.
The City’s best case shows an annual General Fund shortfall of $21.3 million in FY12-13. This is a status quo “baseline budget” with no change in current positions and no COLAs. Without making new, major reductions in the City’s costs, the annual deficit will increase to an average of $34.5 million over FY15-16 through FY20-21. None of these shortfalls can exist because the California Constitution prohibits cities from deficit spending

Current year (2012/2013) budget documents

Budget proposal

Excerpts:

In my budget transmittal memo for last year I mentioned:

“The City of Stockton is near insolvency. Reserves in the General Fund are exhausted, reserves in internal service funds are near depletion and the City has leveraged itself with inter-fund loans and debt that puts the General Fund at risk for backfilling deficits in other funds. Furthermore, the City is saddled with multi year labor contracts with escalating costs that simply cannot be paid within anticipated revenues in the foreseeable future.”

Resources

BBC Archive Statistics

Rufus Pollock - June 22, 2012 in Musings

Some approximate stats about BBC holdings of different types of media from Jake Berger’s presentation today.

  • 2.3 million items of FILM and VIDEO
  • 300,000 hours of AUDIO
  • 1.1 million items of COMMERCIAL MUSIC
  • 4 million items of SHEET MUSIC
  • 4 million PHOTOS
  • 500,000 subject categories
  • 1 million programme contributor NAMES
  • 9 million searchable RECORDS (10 miles of shelves)
  • 25 million PRESS CUTTINGS
  • 70,000 BOOKS
  • 206,899 digitised Radio items
  • 22,965 digitised TV Programmes

DataHub – Small Pieces Loosely Joined

Rufus Pollock - June 22, 2012 in CKAN, DataHub