Data drives the modern organizations of the world and hence making sense of this data andunraveling the various patterns and revealing unseen connections within thevast sea of data becomes critical and a hugely rewarding endeavor indeed. Thereis a need to convert Big Data into BusinessIntelligence that enterprises can readily deploy.
Better dataleads to better decision making and an improved way to strategize fororganizations regardless of their size, geography, market share, customersegmentation and such other categorizations.Hadoopis the platform of choice for working with extremelylarge volumes of data. The most successful enterprises of tomorrow will be theones that can make sense of all that dataat extremely high volumes and speeds in order to capturenewer markets and customer base. Ø 5 V’S OF BIG DATA:- Big Data has certain characteristics and hence is defined using4Vs namely: Volume: The amount of data that businesses can collect isreally enormous and hence the volume of the data becomes a critical factor in Big Data analytics. Velocity: The rate at which new data is being generated allthanks to our dependence on the internet, sensors, and machine-to-machine datais also important to parse Big Data in a timely manner.Variety: Thedata that is generated is completely heterogeneous in the sense that it couldbe in various formats like video, text, database, numeric, sensor data and soon and hence understanding the type of Big Data is a key factor to unlocking its value.Veracity: Knowingwhether the data that is available is coming from a credible source is ofutmost importance before deciphering and implementing Big Data for businessneeds.Value: Last but not least, big data must havevalue.
That is, if you’re going to invest in the infrastructure required tocollect and interpret data on a system-wide scale, it’s important to ensurethat the insights that are generated are based on accurate data and lead tomeasurable improvements at the end of the day. Ø BIGDATA TECHNOLOGIES:- Bigdata is an evolving term that describes any voluminous amount ofstructured, semi structured and unstructured data that has thepotential to be mined for information.Big data technologies are important in providing moreaccurate analysis, which may lead to more concrete decision-making resulting ingreater operational efficiencies, cost reductions, and reduced risks for thebusiness.To harness the power of big data, you would require aninfrastructure that can manage and process huge volumes of structured andunstructured data in realtime and can protect data privacy and security. Some technologies used for big dataanalytics:-1. Hadoop: – Hadoop is an Apache open sourceframework written in java that allows distributed processing of large datasetsacross clusters of computers using simple programming models.
A Hadoopframe-worked application works in an environment that provides distributedstorage and computation across clusters of computers. Hadoop is designed toscale up from single server to thousands of machines, each offering localcomputation and storage. 2. MongoDB: – MongoDBis an open-source document database that provides highperformance, high availability, and automatic scaling. MongoDB obviates theneed for an Object Relational Mapping (ORM) to facilitate development. 3. MapReduce: – MapReduce is a programming model for writing applications thatcan process Big Data in parallel on multiple nodes.
MapReduce providesanalytical capabilities for analyzing huge volumes of complex data. MapReducedivides a task into small parts and assigns them to many computers. Later, theresults are collected at one place and integrated to form the result dataset.4. Hive:- Hiveis a data warehouse infrastructure tool to process structured data in Hadoop.It resides on top of Hadoop to summarize Big Data, and makes querying andanalyzing easy. Initially Hive was developed by Facebook, later the ApacheSoftware Foundation took it up and developed it further as an open source underthe name Apache Hive. It is used by different companies.
For example, Amazonuses it in Amazon Elastic MapReduce. 5. Apache Pig:- Apache Pig is an abstraction over MapReduce. It is atool/platform which is used to analyze larger sets of data representing them asdata flows. Pig is generally used with Hadoop; we can perform all the datamanipulation operations in Hadoop using Pig.I. WHAT IS ASMART CITY?A smartcity is an urban area that uses different types of electronicdata collection sensors to supply information used to manage assets andresources efficiently.
This includes data collected from citizens,devices, and assets that is processed and analyzed to monitor and managetraffic and transportation systems, power plants, water supply networks, wastemanagement, law enforcement, information systems, schools, libraries,hospitals, and other community services.The smartcity concept integrates information andcommunication technology (ICT),and various physical devices connected to the network (the Internet of things or IoT) to optimize theefficiency of city operations and services and connect to citizens. Smartcity technology allows city officials to interact directly with both communityand city infrastructure and to monitor what is happening in the city and howthe city is evolving.Visionof smarter cities:-–Environmental sustainability and efficiency–Sustainable homes and buildings–Efficient use of resources–Efficient and sustainable transportation– Betterurban planning – livable citiesSmartCity Applications:-· Smart parking:Monitoring of parking spaces availability in the city. · Structural Health:Monitoring of vibrations and material conditions in buildings, Bridges andhistorical monuments. · Noise Urban maps:Sound monitoring in bar areas and centric zones in real time.
· Smartphone detection:Detect smart phones and in general any device which works with Wifi orBluetooth interfaces. · Electromagnetic field levels: Measurement of the energy radiated by cell stations and WiFirouters. · Traffic Congestion:Monitoring of vehicles and pedestrian levels to optimize driving and walkingroutes. · Smart lighting:Intelligent and weather adaptive lighting in street lights. · Waste management:Detection of rubbish levels in containers to optimize the trash collectionroutes.
Smart roads: Intelligent Highways with warning messages and diversions according to climate conditions and unexpected events like accidents or traffic jams.