Hadoop Summit: a must go event for people interested in Big Data!

Not so long ago, precisely between 15th and 16th April, for two days Brussels was a world capital of new technologies. And all this because of the Hadoop Summit conference in which I participated in as a BI/Big Data Consultant and a representative of Apollogic.

Hadoop Summit, organised by Hortonworks, is one of the largest world conferences devoted to the Hadoop ecosystem. Global Big Data ideas and major achievements were presented during the conference. This year’s edition of the summit, which took place in Brussels, proceeded under the major heading of “Hadoop is enterprise ready”. To prove the point, examples of hundreds of companies were presented which have been building their applications and systems on the foundation o the Hadoop ecosystem. Among those companies were such suppliers of software as SAP, Microsoft, Teradata, Hortonworks, MapR, Cloudera, Pivotal, SAS, Informatica or Cisco,  or Internet giants such as Google, Yahoo, Pinterest or Spotify, and many companies using Hadoop in their business operations, such as British Gas ING,  and many others.

The conference was a platform for exchanging information and experience with technology suppliers who had their stands there, but the most important elements were specific subject panels, where everyone could find something interesting. The panels were open and, therefore, you could choose the most interesting subjects from one of the six following paths:

• Committer Track
• Data Science & Hadoop
• Hadoop Governance, Security & Operations
• Hadoop Access Engines
• Applications of Hadoop and the Data Driven Business
• The Future of Apache Hadoop

Apart from the main message of the conference, “Hadoop is enterprise ready”, many themes touched upon new trends in IT such as: Rebel Data Warehouse, DataLake and the Internet of Things. During lectures, one could learn how to start building a new, limitless data warehouse from scratch, using a scalable technology, and reduce the costs of data storage to unimaginably low price levels. The absence of limitations means not only an unlimited capacity of the data warehouse but also the possibility of delivering interfaces which make it possible to access data and analytical tools. Thanks to such solutions, using the Internet of Things will become simpler than ever; it is going to be easier to detect financial cheats, to make the roads safer facilitate many other things.

Thanks to  IoT, it will be possible to talk about turning the Internet around: until now the Internet was based on receiving information by millions of consumers from single servers, today information from millions of sources end up in single servers. The amount of data collected in such a way, and development of the analytical sciences make single servers incapable to offer enough calculating power and space in data warehouses, while Hadoop ecosystem tools offer stable and ready-made solutions, based on an open source technology.

The subjects discussed during the main session of the conference included usefulness of the solutions in everyday life of average people. During his speech, Ben Hammersley pointed out that all technologies are expected to serve the people, and to simplify life. He started his speech from uttering the following words: “We won. Now we have a problem”. What he meant by that was that technology today has extended its limits much more than we had imagined, and nowadays it is not optimising speed, or increasing efficiency, developing data analysis and machine learning because we have got all that – the most important thing is to adjust the technology to our needs and expectations. One of the ideas that I remembered particularly well was building intelligent cities. In New York or London intelligent cities are supposed to, first of all, optimise the amount of time during which you can reach one spot in town from another, but there are also areas where people do have time and, riding bicycles, can admire the beauty of nature. What is important for them, more than anything else, is to meet as many colourful and smiling people as possible and, on their way to work, in their favourite coffee-shop, they could have a cup of coffee without waiting in the queue. Today, this is how we should optimise the world!

This year’s edition of the Hadoop Summit confirms that active utilisation of the Big Data technology at Apollogic was a good move. It proves that the direction of development of our products, such as ApoSentinel. The event was very inspiring, and in the nearest few months our products will be enriched with new functionalities which will revolutionise not only the technological world.

This report from this exceptional event is from Marcin Siudziński, our BI/Big Data consultant at Apollogic.