The data units of urban construction include

The advancement rate of these units is 1024 (2 to the tenth power)

Wikipedia definition

Definition of research institution

McKinsey Global Institute Definition

Hadoop was born in 2005

At the end of 2008, big data was recognized by some well-known computer science researchers in the United States.

In 2009, the Indian government established a biometric database for identity management, and the United Nations Earth Pulse project has studied how to use data from mobile phones and social networking sites to analyze and predict treponemal disease outbreaks.

In February 2010, Kenneth Cukier published a 14-page special report on big data in The Economist, "Data, Ubiquitous Data"

In March 2012, the Obama administration released the "Big Data Research and Development Initiative" on the White House website. This initiative marked that big data has become an important feature of the times.

In April 2014, the World Economic Forum released a global information technology report with the theme of "Returns and Risks of Big Data"

The total amount of global data in 2017 was 21.6 GB. The current annual growth rate of global data is about 40%. In 2018, the global big data industry has developed strongly.

Large size

high speed

diversification

Low value density.

(1) Scientific research produces big data.

(2) Internet of Things technology generates big data.

(3) Networking generates big data.

The Internet of Things, mobile Internet, social networks and e-commerce are the application forms of the new generation of information technology. These applications gradually form and generate big data during their operation.

With the continuous development of big data and related technologies, new products, new technologies, new business formats and new services for the big data market are gradually emerging and developing rapidly.

With the development of information technology, more and more industries have entered the track of transformation and development, and corporate decision-making has gradually shifted from business-driven to data-driven. Big data analysis can support companies to launch more effective and standard marketing plans. Able to develop personalized service measures for enterprises that are more in line with consumer needs.

The impact of big data and related technologies on scientific research is increasingly apparent

According to the 2018 Global Digital Report, there were 3.196 billion global social media users in 2018. A year-on-year increase of 13%.

The so-called computer is actually the working windows

For the monitoring program, this windows does not deal directly with the hardware, and all resources need to be scheduled by the monitoring program. Allocation, so this Windows machine is a virtual machine, referred to as vm. This process of separating the hardware machine from the operating system through the monitoring program is virtualization.

resource pool

secondary virtual

User self-management.

Connect to the network.

Private Cloud

public cloud

hybrid cloud

Infrastructure and services

Platform and services

Software and services

others

Cloud computing is a pay-per-use model that provides convenient, on-demand network access to a pool of configurable computing resources. These resources can be provisioned quickly with minimal administrative effort or interaction with the service provider.

Baidu cloud

Ali Cloud

Amazon aws cloud service.

Microsoft azure cloud computing.

The cloud computing architecture is divided into 4 layers, the physical resource layer, the resource pool layer, the management middle layer and a

Perceptual layer

transport layer

data layer

Application layer

terminal layer

Distribution

Transparency

unity

homogeneity

GFS

HDFS

Ceph

Luster

Distributed applications and services

Distributed static resources

Distributed data and storage

Distributed Computing

Distribution refers to deploying different service modules in multiple different servers, working together through destiny calls, and providing services to the outside world.

Clustering refers to deploying the same application or service module on multiple different servers, and a cluster is used together. Balanced equipment provides services to the outside world.

Dong Cunting, the creator of the project, explains how Hadoop got its name: "This name was given by my children to a well-fed brown elephant."

The initial version (now called IDFS I MapRediuce) was implemented in 2004 by Dorug Cutting fu Mike Catiarcla.

December 1, 2005 - Nutch was transplanted to the new framework, and Hadoop was running stably on 20 nodes.

In February 2006, the Apache Hadoop project was officially launched to support the independent development of MapRcduce and HDFS.

In February 2006, Yahoo's grid computing team adopted Hadoop.

April 2006 standard sort (10 GB per node) ran in 47.9 hours on 188 nodes.

In July 2007, the research cluster reached 900 nodes.

In April 2008, the research cluster loaded 10 TB of data every day.

April 2009 - Win Sorts per Minute, sort 500 GB in 59 seconds (on 1400 nodes) and 173 points

The distributed file management system of the Hdfshadop platform is the most important of ap. One of the components

Natanode is the execution of specific tasks.

Interactions among Danodenamenoe

Maprduce is the core computing component of adop and is used for parallel computing of massive data. The core steps of the Mapreduce framework are mainly divided into two parts, Map and reduce

Big data is not only suitable for large enterprises, but also for enterprises of all different sizes.

Provides a foundation for the esoteric enterprise and critical insider knowledge of external data for Big Data Hadop.

Hadop has quickly grown to become the preferred big data analytics solution for unstructured data.

Google and hadop are inextricably linked.

IBM announced the creation of a new storage architecture as a cluster on AP. Runs DB2 or ACL databases to enable applications to support high-performance analytics, data warehouse applications and operational computing.

Internet search giant Baidu is also considering using AP

As a distributed infrastructure, Ap allows users to develop distributed programs without understanding the underlying details of distribution.

Adjust mpreduce strategy.

Improve HD FS efficiency and functionality.

Modify the execution strategy of speech ative.

Control resource usage.

Platform design drawing

Make full use of the massive log data accumulated on the mobile terminal.

Conduct big data analysis on user usage and advertising business.

Utilize Alibaba Cloud's big data technology to support the operations team and advertising team in optimizing existing business based on the analysis of log data.

We will continue to develop weather monitoring based on mature big data technology and big data talent resources.

Moji will continue to expand the breadth of weather monitoring services.

In the early stage, it has a huge impact on the market.

Multiple interactivity has been greatly improved through the mobile Internet.

Commercial meteorological services will be a huge entrepreneurial opportunity in China.

The Internet of Things refers to the deployment of various devices with certain sensing capabilities, computing capabilities and execution capabilities in entities in the physical world. Information sensing equipment realizes information transmission, collaboration and processing through network facilities, thereby realizing wide-area or large-scale communication between people and people. The interconnection of information exchange needs between things and things.

data collection

data storage

Statistical Analysis

NB-LoT technology

RFID technology

Sensor Technology

Network communication technology

Embedded system technology

cloud computing