Tag Archives: disasters

Microsoft releases 18M building footprints in Africa to enable AI Assisted Mapping

In the last ten years, 2 billion people were affected by disasters according to the World Disasters report 2018. In 2017, 201 million people needed humanitarian assistance and 18 million were displaced due to weather related disasters. Many of these disaster-prone areas are literally “missing” from the map, making it harder for first responders to prepare and deliver relief efforts.

Since the inception of Tasking Manager, the Humanitarian OpenStreetMap Team (HOT) community has mapped at an incredible rate with 11 million square kilometers mapped in Africa alone. However, large parts of Africa with populations prone to disasters still remain unmapped — 60% of the 30 million square kilometers.

Under Microsoft’s AI for Humanitarian Action program, Bing Maps together with Microsoft Philanthropies is partnering with HOT on an initiative to bring AI Assistance as a resource in open map building. The initiative focuses on incorporating design updates, integrating machine learning, and bringing new open building datasets into Tasking Manager.

The Bing Maps team has been harnessing the power of Computer Vision to identify map features at scale. Building upon their work in the United States and Canada, Bing Maps is now releasing country-wide open building footprints datasets in Uganda and Tanzania. This will be one of the first open building datasets in Africa and will be available for use within OpenStreetMap (OSM).

In Tasking Manager specifically, the dataset will be used to help in task creation with the goal of improving task completion rates. Tasking Manager relies on ‘ML enabler’ to connect with building datasets through an API. This API-based integration makes it convenient to access not just Africa building footprints, but all open building footprints datasets from Bing Maps through ML Enabler, and thus the OpenStreetMap ecosystem.

“Machine learning datasets for OSM need to be open. We need to go beyond identifying roads and buildings and open datasets allow us to experiment and uncover new opportunities. Open Building Dataset gives us the ability to not only explore quality and validation aspects, but also advance how ML data assists mapping.”
– Tyler Radford (Executive Director, Humanitarian OpenStreetMap Team)

Africa presented several challenges: stark difference in landscape from the United States or Canada, unique settlements such as Tukuls, dense urban areas with connected structures, imagery quality and vintage, and lack of training data in rural areas. The team identified areas with poor recall by leveraging population estimates from CIESIN. Subsequent targeted labeling efforts across Bing Maps and HOT improved model recall especially in rural areas. A two-step process with semantic segmentation followed by polygonization resulted in 18M building footprints — 7M in Uganda and 11M in Tanzania.

Extractions Musoma, TanzaniaExtractions in Musoma, Tanzania

Bing Maps is making this data open for download free of charge and usable for research, analysis and of course, OSM. In OpenStreetMap there are currently 14M building footprints in Uganda and Tanzania (the last time our team counted). We are working to determine overlaps.

We will be making the data available on Github to download. The CNTK toolkit developed by Microsoft is open source and available on GitHub as well. The ResNet3 model is also open source and available on GitHub. The Bing Maps computer vision team will be presenting the work in Africa at the annual International State of the Map conference in Heidelberg, Germany and at the HOT Summit.

– Bing Maps Team

Go to Original Article
Author: Microsoft News Center

LTO-8 tape standard to provide insurance against disaster

This summer’s weather disasters should have been a wake-up call about the need for a sensible business continuity strategy. But even now, several months later, recovery from flooding and infrastructure damage remains a slog.

Those who contented themselves with the nonsense that high-availability architecture is a substitute for good, old fashioned disaster recovery planning are behind the eight ball. So, too, are companies that used “cloud-based” disaster recovery services that were actually managed hosting shops located a few miles from their offices and subject to the same disaster.

The good news is technology does exist to make reliable copies of data and get them to distant off-site storage in an affordable way. I’m talking about tape, of course: Linear Tape-Open standard tape. It’s compatible across virtually all libraries, regardless of manufacturer, and growing in terms of speeds and feeds, durability, longevity and capacity.

Some vendors are already talking about generation LTO-8, though LTO-7 hasn’t run its course, and the LTO-8 tape drive isn’t yet in mass production. Some say that LTO-8 caters to a niche market of large enterprises and industrial clouds. By contrast, LTO-7 may be the last iteration catering to small to medium-sized customers given its price point of about $100 per 6 TB cartridge (15 TB compressed).

LTO-8 not a niche market

The increased capacity of the LTO-8 tape drive, at 12.8 TB native and 32 TB compressed, and data throughput of 472 Mbps native and 1,180 Mbps compressed, should garner “No h8 for eight.” (That’s my meme, but you can use it if you want.) As for the “niche” thing, I must disagree.

The preponderance of data we generate today isn’t the stuff of big data analytics.

First off, no business measures data storage in terabytes anymore. Even small ones handle hundreds of terabytes, or even petabytes, of storage. And larger companies that used to complain storage was out of hand in the tens of petabytes range are easily into exabyte-sized repositories today. The largest data producers and storage firms, according to IDC and others, exceed tens of zettabytes now, with analysts predicting 163 ZB of new data a year by 2025.

So LTO-8, with its 32 TB capacity, seems to be just what the doctor ordered for companies most likely to make big use of tape technology: cloudies and data-intensive verticals, such as healthcare, surveillance, research labs, and oil and gas. These firms are putting tape back to use in an old, secondary storage role.

Tape’s special role

I know you’ve heard that tape doesn’t meet the needs of active archives. I wrote a column on this topic recently for Storage magazine, and the answer is “Maybe, maybe not.” But the truth remains that the preponderance of data we generate today isn’t the stuff of big data analytics. Clouds want to sell “a big storage repository in the sky” to host all of the seldom accessed and never modified data consumers create with their million-megapixel phone cameras. Tape is the go-to technology for virtually all of those services now.

And getting large quantities of data to the cloud from the real world is going to require something besides expensive wires and fiber optic cables. Tape is a great mechanism for data transport without a WAN or metropolitan area network.

Eight is a special thing

Truth be told, the eight in LTO-8 is rich as a moniker. In the hard sciences, eight has many applications, ranging from being the method for describing particles in particle physics to the second magic number in nuclear physics. Eight is the number of vertices in a cube in geometry, and it’s the first number that’s neither prime nor semiprime in mathematics. It’s the atomic number of oxygen, and in biology, it’s part of the definition of the Arachnida and Octopoda classes of life. And, of course, in computer science, it’s the foundation of the numbering system used to describe digital information itself — 8 bits to a byte.

Eight holds meaning as the intersection of natural and supernatural among those who study numerology and religion. The Chinese equate eight with money and luck, spending inordinate amounts of money to obtain phone numbers with many eights in them. The old saw about the Beijing Olympics, that the opening ceremonies began precisely on Aug. 8, 2008, at eight minutes and eight seconds past 8 p.m. local time (Coordinated Universal Time+8), shows how important the lore of eight remains among a large segment of the world’s population. Marketeers cater to such things. General Motors, for example, calls the minivan built for the Chinese market the Buick GL8, but gives the same product a different name in other countries.

To numerologists, eight is a karmic number, “What goes around comes around,” as eight looks like the infinity symbol turned sideways. It symbolizes the accomplishment of goals while remaining generous and willing to take risks. In the religious realm, Eastern arts define eight paths to good or heaven, while Western faiths ascribe a lot of natural and supernatural interfaces to eight.

Whether or not you buy into the metaphysics of eight, LTO-8 does seem to have come at a fortuitous moment. We need its abundance — another meaning ascribed to eight, in Hinduism — and we need the luck that it might bring the next time a hurricane or other calamity strikes.

At a minimum, the luck of LTO-8 is tied to what it can do to lower the costs of storing zettabytes of data. A massive, multi-zettabyte tape repository will likely cost less than the energy costs alone for a repository with the same capacity based on disk or flash. The release of LTO-8 will provide yet another watershed in data storage capacity and cost.

Author’s note: In my October column, I incorrectly reported that Quantum had departed from the tape storage market. Quantum continues to sell tape storage. I regret the error. Look for a deeper discussion of Quantum’s outlook on tape technology and archiving in an upcoming column.