Mapping A City
My friend Ro Gupta co-founded and runs a company called Carmera which makes real-time HD maps for the autonomous driving market.
They do this by operating “an owned and professionally crowdsourced vehicular sensor network” (ie Google StreetView) which captures real-time data about what is happening at street level.
In an announcement they made today, they describe a partnership with the City of New York in which they “will share data with the NYC Department of Transportation, including historical pedestrian density analytics and real-time construction detection events. In turn, we’ll gain access to key city data sets that allow all parties to work together to improve the accuracy of street inventories while doing our part to increase private-public transparency.”
This is an example of a pedestrian density map of the west village in NYC, where we live.
This is an example of a growing trend I am seeing where tech companies, which traditionally have wanted to “do it ourselves”, are partnering with the broader ecosystem (large companies, governments, public agencies, etc) to build more comprehensive data sets.
It is also an example of goverments and other large bureaucracies getting more comfortable sharing their data and being more open to working with startups and the broader tech sector.
This is a very hopeful trend.
We have significant problems to solve in the coming decades around a warming planet, overloaded and failing transportation systems and urban infrastructure, and many other things.
If all sectors of society come together to work on these problems, we stand a much better chance of solving them.
Comments (Archived):
https://uploads.disquscdn.c…Too many dogs in urban settings
Thanks, Fred. Figuring out how to work with govs was something we didn’t initially know much about and had to learn — and we had a lot of help from orgs like Tech:NYC and EDC. I’m sensing a bunch of our startup peers are on a similar learning curve. We’re still learning but this is an encouraging start.
Congrats Ro. Curious how do you incentivize users to share/participate?What do you think of blockchain models like https://foam.space/ for example, where the token is used by users to stake their participation and sharing of location data?
Thanks, William. We don’t really do crowdsourcing for spatial data in the traditional sense — although definitely rooting for some of these new blockchain experiments. We tend to need a lot more control over coverage, freshness and quality, so we partner with professional fleets who’s operations fit our needs, and provide them a safety analytics and monitoring service in return for helping us gather this data efficiently through sensors we install on their vehicles. Our little term for this is “pro-sourcing”; sort of a B2B twist on pure consumer crowdsourcing.
Got it. Thank you
Nice to see your face pop up Ro!
Nice. Congratulations!
Go Ro Go!
Yea Ro!
Ro – you the man! Your success growth and funding is so fun to see. Super impressive what you’ve accomplished.
The entire world’s comfort with the web and information distribution will lead to new structures.P3s will be a big part of it.
i’m not clear how this pedestrian density data (the map above) is being collected in the west village. what’s the tech, anyone?
100% my guess…but…it sounds like it’s cameras on cars driving around – the pictures then get auto-tagged and analyzed…and so that’s probably how they are detecting how ‘crowded’ a given street or corner is for something like foot traffic.
thanks. i wonder how often the cars snap the same corner (rhetorical thought)? sounds a bit hit and miss (lacking accuracy). i’m surprised mobile phone data connections to cell network nodes aren’t being used in real time to capture density.
Well aligned with today’s announcement of Bloomberg Philanthropies’ launch of Results for America, an initiative that helps cities use data and facts to tackle their most pressing challenges and improve residents’ lives.. https://www.nlc.org/article…
Thanks for sharing Fred! This is great! I live on Jones St in the West Village (between Bleeker and w 4th) is there anyway to zoom in?? Would love to see the activity around my building at all hours.
ForWe have significant problems to solve in the coming decades around a warming planet, overloaded and failing transportation systems and urban infrastructure, and many other things.in particular, forWe have significant problems to solve in the coming decades …Yup.But for… a warming planetNope:(1) There’s no, none, nada, nil, zip, zilch, zero evidence that the planet is “warming” in any significant sense.(2) If the planet were warming, all the solid evidence is that humans could do nothing about it.(3) Worrying about a “warming” planet is down on the level of the uninformed, irrational, superstitious, irresponsible, neurotic nonsense that got the Mayan people to let the Mayan charlatans kill people to pour their blood on a rock “too keep the sun moving across the sky”, of the charlatans in various manipulative religions that have had the people “sacrifice” sheep, goats, cattle, children, etc. to, say, get redemption from the retribution of their transgressions, of the quasi-religion of earth worship of charlatan Saint Laureate Al Guru and his followers, e.g., Obama, out to get us to sacrifice, e.g., our cars and have our electric bills “necessarily skyrocket”, of the manipulative NYT to establish in NYC a tribal norm of fear of “warming” to hook their gullible readership eager to fit in and, thus, get more readers, eyeballs, and ad revenue, of the people who want to get subsidies for wack-o projects for wind, solar, etc. energy, shutdown the US fossil fuel industry, get regulations to attack the fossil fuel industry and, thus, “squeeze” them for money, power, etc.Easy to see why the overly emotional, irrational, irresponsible, self-destructive NYT pushes this nonsense.But the solution is easy: Tell the NYT just to shut up, sit down, listen up, stop their irresponsible mass insanity, fold up, and go away. In the meanwhile, conclude that on paper the NYT can’t compete with Charmin and on the Internet is useless for wrapping dead fish heads and just IGNORE it.Simple.I’m shocked that the supposedly relatively knowledgeable and rational audience at AVC.com would entertain for even a nano second this NYT neurotic manipulation.For other problems, the leading one is, our country is so bad off, necessarily about the worst in all of history, although now significantly better than under the NYT god Obozo, that we are not reproducing and, instead, are rapidly going extinct. And IIRC ~50% of the babies we are having are on public assistance. Humanity made it through the ice ages but is failing due to the neurotic NYT. Happy Thanksgiving eating alone, gee, while reading the NYT.Right: So in face of going extinct, we embrace the irresponsible, neurotic NYT, continue to shoot our family formation in the gut, and f’get about what is important.Go for it NYC! Go east! About 500 miles east. Then go down, about 3 miles down. SF and Hollywood, do much the same except go west. Then maybe the rest of the country can get back to rationality, family formation, etc.Here’s a guess at what is going on: The Democrats under Obozo, and they hoped would have under Hildebeest, had a good thing, national version of Chicago pay-off, kick-back, patronage, machine politics, going and, now, with Trump in the White House want to get back to that.The Democrats have no meaningful rational arguments: What are they going to say? Admit that they want to throttle fossil fuels, subsidize wind and solar, thus, get payoffs from both sides while US electric power rates “necessarily skyrocket” and the US economy is on the way to Kenya?So, the Democrats are borrowing from Nazi Dr. J. Goebbels and his “If you tell a lie often enough, then people will believe it.” But there is a Democrat and NYT, ABC, CBS, CNN, MSNBC, NBC, WaPo update:”If constantly spout total nonsense, absurd, wild accusations with no evidence, ignore rationality and solid evidence, and keep this up 24 x 7 for years, then eventually people will give up looking for evidence and give up on rationalism, become willing to tolerate the intellectual sewage, and be willing to set aside their own interests. Madness will rein. Then the Democrats and the NYT, etc. will have a chance against Trump and for a return to Obama-Hillary insanity.”Heck, for years the NYT has been getting away with such a descent into mass NYC, NYS, and NE and West Coast irrationality scam with their absurd screaming about “global warming”. So, just continue it: Have people just flatly stating, 24 x 7, with no evidence, no professional qualifications, no references that Trump is “unfit” to be POTUS.The scam is to suggest”What are you going to believe? The Democrats, Pelosi, Hillary, Fauxcahontas, Whoopi, Behar, O’Donnell, Obozo and the NYT or just your own lying eyes? Do you really trust yourself to take evidence seriously, or wouldn’t you really rather just fit in with the rest of the tribe?”The media has often been outrageous garbage way back to Jefferson’s scathing statement. And, it’s long been easy to see that the media concentrates on scaring people and using the techniques of formula fiction. So, they keep the average level of quality and rationality low and, thus, easy to meet. Putting out well researched, referenced, e.g., even as just in common high school term paper writing standards, would raise the bar of what they could print and, thus, make their work harder, their publications shorter, etc. So, low grade, emotional, irrational nonsense, short on facts, references, and rationality rule. Now with the NYT, Obozo, and the Democrats, the situation is much the same but much worse. It does appear that nearly all the media wants to get the public used to garbage and give up on information.Here’s about the last straw. Here is a picture of two people:https://media.breitbart.com…The one on the right is an illegal alien in the US, IIRC, a confessed murderer, and just murdered the one on the left. The one on the left is a US citizen, 20 year old college student from a good family in Iowa and just murdered by the one on the right.The Democrats care a lot about the one on the right and not at all about the one on the left.So, in response, Fauxcahontas screams that Trump is “separating families”. The one on the right “separated a US family”. The families that ICE, etc. are separating are illegal alien criminals. That’s what our law enforcement system does with criminals — puts them in jail and, thus, separates families. We do this with US citizens. We sure as heck don’t owe family sized hotel suite vacations to illegal alien criminals. But Fauxcahontas and the Democrats and their media buddies keep pushing their irrational, irresponsible insanity that somehow illegal alien criminals deserve protection, welfare, and rights of and beyond those of law abiding US citizen families. Insanity rules.We need The Wall. We need ICE. Trump wants The Wall and ICE, and the Democrats are strongly against both — want NO WALL and want to abolish ICE.Degenerate, disgusting insanity rules.At least my server works, my startup software math works, my startup software works, I’m collecting data for my startup, and last night while going to sleep noticed that can clean up some of what the MIT does with math in quantum mechanics — just do a short derivation (in my head) about L^1 and L^2 (here the L abbreviates Lebesgue, that is, Henri, Emile Borel student, about 1900) and then use the classic Riesz–Fischer theorem; darned nice theorem, surrounded by an ocean of bizarre, pathological counterexamples that don’t quite meet the assumptions of the theorem! So, use the L^1 assumption and a little derivation to justify the L^2 assumption of the theorem — uh, for complex valued function f of real x, L^1 asks that the integral of |f(x)| be finite (and quantum mechanics assumes that it is due to |f(x)| being essentially a probability density) and L^2 asks that the integral of |f(x)|^2 be finite. That’s a start, but MIT’s quantum mechanics needs a lot more cleanup than that. At least such math is back to some rationality.I’m going to get some exercise.Update: Got an hour of exercise and am soaked with sweat.Get ice water and/or lemonade and get some more exercise!
I would use every ounce of portland cement destined for the wall to build a huuuuge pyramid with a golden top and put comrade Donald inside, for good.He is completely nuts and a liability for the galaxy.Left or right doesn’t matter anymore.
ForHe is completely nuts and a liability for the galaxy.AI believe Trump is TERRIFIC. Why:(1) He has the US economy going again: (A) GDP growth rate way up since the Obama years, now some quarters above a rate of 4% per year. (B) Unemployment rates the best ever for Blacks and Hispanics and the best in decades for women. (C) Apparently significant progress in the number of people employed. (D) Both interest rates AND the inflation rate surprisingly low.(2) There’s a LOT more to do to get more of the people able to work back to work, but he seems to be working on that: (A) Trade deals and import tariffs seem to be generating more hiring, including in manufacturing. (B) He claims to be having his HUD Secretary Dr. Carson working on vocational training in poor neighborhoods for new jobs, maybe via Foxconn. (C) He seems to be slowing the flood of imported HxB workers that have been putting US citizens out of work.(3) He is working hard to get the illegal alien criminals OUT of the US: (A) He has ICE rounding up the criminals and deporting them. (B) He is working hard with the limited means he has to stop the flow of illegal alien criminals crossing into the US from Mexico.(4) Apparently he has been working hard on trade deals: (A) He keeps telling China that the several hundred billion dollar a year deficit can’t last, and has been putting on tariffs. Supposedly Trump and Xi are to meet again to work out some “fair, reciprocal” trade without a destructive “trade war”. (B) He claims progress with Mexico on NAFTA renegotiation. (C) He is pressuring Canada on letting the US sell farm products to Canada and slow the cheap softwood lumber from Canada to the US. (D) He got the US OUT of the Trans Pacific Partnership (TPP), where the rules were set with too little input from the US.(5) Apparently he has throttled the tree-hugging, Greenie, earth-worshiping, guilt-ridden, irresponsible, highly destructive, hot bed of EPA quasi-religious total wackos. (A) He has the US back in the fossil fuel business. E.g., he has fracking going full blast in the Permian Basin, etc. in Texas and in other parts of the US. He is exporting natural gas and crude oil. (B) He has coal going again, with the additional jobs, GDP, business, etc. (C) He got the US OUT of the UN, IPCC, total wacko, highly destructive, really economic sabotage, Paris Accords. (D) He has approved the various pipelines needed to get oil from Canada and the Dakotas to the refineries in Texas. (E) He has US steel going again, USS and others opening lots of plants, some new.(6) Supposedly he has done a lot to get rid of the wacko regulations: E.g., apparently get funds from company B to have regulations that help company B and choke their competitor company A. Then get kickbacks, payoffs, etc. from both A and B. It appears that Trump has stopped this dirty Obozo machine politics crap.(7) Apparently under Obozo much of the top of the US DOJ and FBI had become a total Democrat Party attack machine and influence peddling buddy system. Trump is letting the dirt come out, and likely after the election will (A) fire the worst of the nonsense at both DOJ and FBI, (B) put in good leadership, (C) shut down the Mueller witch hunt, (C) bring the obvious cases for the astounding law violations of various politicians over the past few years.(8) He has Melania being a fantastic symbol of US caring and pride.(9) He has the US military going again. It appears that he has a crash program for defense against ICBMs, e.g., from North Korea or Iran.(10) The Jews of Israel and the US should be happy about Trump’s move of the US embassy to Jerusalem.(11) North Korea and Iran are threats of ICBM launched nukes, and Trump has been pushing back hard with sanctions and negotiations, and so far he has not had to fire a shot.(12) Trump promised to “bomb the shit out of ISIS”; well, Secretary Mattis did something that did work quite well. The land ISIS occupied has been freed, and there is not much left of ISIS.(13) Trump is getting along very well with the Gulf States, especially the Saudis.(14) Trump has the NATO countries starting to pay as they were supposed to.(15) Far too much of the US mainstream media (MSM), e.g., NYT, ABC, CBS, CNN, MSNBC, NBC, WaPo, Boston Globe, have been tossing out nearly 100% total made up, cooked up, totally false and “fake”, nonsense accusations — e.g., remote psychiatric diagnoses with no professional credentials or patient examination — 24 x 7, but apparently so far the worst accusation against Trump since he was elected that has any credibility is that at some dinner he had an extra scoop of ice cream. Since that is the worst, Trump gets a world-class, A++ clean bill of health.And there’s more.So, I’m very glad for Trump, and am starting to conclude that we will have to go back to Lincoln or Washington to find a better POTUS.But I don’t want to be wrong: If you have some credible evidence of anything significantly wrong with Trump, then please trot it out.
The trend towards sharing data opens the question of its standardization which makes for better sharing, and its tagging/metrics.We just came back from a conference at NASA on biomimicry, and after presenting our biological and pattern systems, we joined clusters to solve for this kind of open source private public partnership of large open data sets. Making data sets open for usage, makes them more available (say medical or education info) when masked, and yet still, sets of standards make using the info still clumsy and cumbersome, an example being metric vs imperial.We will be part of visual/class tagging solving with groups there around this and machine learning, so while it’s exciting to see this open up to these collaborations, I’m still curious as to how we efficiently mount the hurdles so the data is less work to deal with. Any thoughts?
This is a great trend. Am particularly excited to see how it can be used to help solve environmental issues.
Friend of mine is building virtual road networks to train autonomous vehicles. Huge training set.
Thanks for sharing – Great example of the emergent practice of data collaboratives – We have tried to capture and learn from 100 + examples and translate that into a methodology of how to set up these new public private partnerships toward solving public problems using privately collected data – See http://datacollaboratives.org/ –
Thank you Fred for calling this out. It’s great to see this breakthrough public-private partnership getting traction. Solving public problems using privately collected data (and getting past the inertia and roadblocks that abound while trying), is a core challenge and opportunity for our generation.If you’ve not already read it…https://www.mckinsey.com/industries… McKinsey’s Smart Cities is a worthy read that complements this.Final point: This kind of heatmapping could also positively impact hyper-local urban neighborhood and community enhancement efforts. As retail stores empty out and are left vacant with the onslaught of Amazon (and neighborhoods suffer as a result)…this kind of data has potential public and private sector value in assessing foot traffic patterns and densities for re-use of some of these vacant store fronts. (re-use and neighborhood development/enhancement efforts).Todd Bidawww.neytenda.com https://medium.com/@toddbidahttps://uploads.disquscdn.c…