[CivicAccess-discuss] Open Data Consulting Firm

[Glen Newton]

Big data is now at least terabytes and for many big data people, only
starting at petabytes.
And we are usually talking about a single dataset, not some collection
of disparate data.

"Big data[2] are datasets that grow so large that they become awkward
to work with using on-hand database management tools. ....Though a
moving target, current limits are on the order of terabytes, exabytes
and zettabytes of data." -
https://secure.wikimedia.org/wikipedia/en/wiki/Big_data

Government seldom has this, except for sensors (like remote sensing,
climate, rain gauge and other sensor), modelling, and scientific work
they might be doing.

StatsCan data (collections of data, that is) are probably just a
couple of hundred GBs, which hasn't been big data for at least a
decade.
NRCan has remote sensing data, and is involved in the some other
sensor projects producing Big Data (Neptune?
http://www.neptunecanada.ca/) that are low end Big Data.
CRA data (collections of data, that is) are also probably in the 100s
of GB size
NRC probably has some data sets that can be considered Big Data.

-glen

On Mon, May 16, 2011 at 10:36 AM, Tracey P. Lauriault
<tlauriau at gmail.com> wrote:
> I did not mean short sighted, just different lenses!
> If you move to geomatics, NRC, NRCan, StatCan, CRA, etc.  You start getting
> into some big data.
>
> On Mon, May 16, 2011 at 10:28 AM, James McKinney <oxford.tuxedo at gmail.com>
> wrote:
>>
>> If big data is relevant to government data, people may be interested
>> in the Strata Conference http://strataconf.com/ There is a free online
>> conference coming up. If there is any doubt people can build a
>> business on data, search for "business intelligence". For big data
>> specifically, search for who's using Hadoop [1].
>>
>> However, I'm not sure how much "big data" there is in government. Big
>> data typically refers to datasets that are either at least hundreds of
>> gigabytes or millions of rows in size (this is a generous lower bound
>> - it is usually much higher). Databases of 311 and 911 calls are
>> potential candidates. Toronto, our biggest city, generates only a
>> quarter million 311 calls per year according to the data they've made
>> available [2]. I don't think open data people are being short-sighted;
>> I think there is just not that much big open data.
>>
>> 1. http://en.wikipedia.org/wiki/Apache_Hadoop#Other_users
>> 2.
>> http://www1.toronto.ca/wps/portal/open_data/open_data_item_details?vgnextoid=3cdebe037654f210VgnVCM1000003dd60f89RCRD&vgnextchannel=6e886aa8cc819210VgnVCM10000067d60f89RCRD
>>
>> On Mon, May 16, 2011 at 9:56 AM, Tracey P. Lauriault <tlauriau at gmail.com>
>> wrote:
>> > but it is interesting nonetheless and it includes a focus on open
>> > source,
>> > highlights the not so open of the word open and also inspires the idea
>> > that
>> > it is possible to do business.
>> > Finally, this company is grappling with big data issues, which many in
>> > the
>> > open data world, primarily in cities, are not even thinking about.
>> >
>> > On Mon, May 16, 2011 at 9:53 AM, James McKinney
>> > <oxford.tuxedo at gmail.com>
>> > wrote:
>> >>
>> >> The name has "Open" and "Data" in it, but it does not seem to have to
>> >> do with "open data".
>> >>
>> >> On Mon, May 16, 2011 at 8:04 AM, Tracey P. Lauriault
>> >> <tlauriau at gmail.com>
>> >> wrote:
>> >> > http://opendatagroup.com/about/
>> >> >
>> >> > --
>> >> > Tracey P. Lauriault
>> >> > 613-234-2805
>> >> > http://traceyplauriault.ca/
>> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > CivicAccess-discuss mailing list
>> >> > CivicAccess-discuss at civicaccess.ca
>> >> > http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>> >> >
>> >> _______________________________________________
>> >> CivicAccess-discuss mailing list
>> >> CivicAccess-discuss at civicaccess.ca
>> >> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>> >
>> >
>> >
>> > --
>> > Tracey P. Lauriault
>> > 613-234-2805
>> > http://traceyplauriault.ca/
>> >
>> >
>> >
>> > _______________________________________________
>> > CivicAccess-discuss mailing list
>> > CivicAccess-discuss at civicaccess.ca
>> > http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>> >
>> _______________________________________________
>> CivicAccess-discuss mailing list
>> CivicAccess-discuss at civicaccess.ca
>> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>
>
>
> --
> Tracey P. Lauriault
> 613-234-2805
> http://traceyplauriault.ca/
>
>
>
> _______________________________________________
> CivicAccess-discuss mailing list
> CivicAccess-discuss at civicaccess.ca
> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>



-- 

-