"Artificial Intelligence without Big Data Analytics is lame, and Big Data Analytics without Artificial Intelligence is blind." Dr. O. Aly, Computer Science.
Introduction: For decades, the traditional database such
as MySQL, PostgreSQL, SQL Server and Oracle are regarded to be a one-size-fits-all approach for data persistence
and retrieval (Sakr & Gaber, 2014). However, these traditional
databases are challenged by the
increasing demand for scalability, the requirement for new applications, and
some web-scale applications (Sakr & Gaber, 2014).
The most common architecture to develop enterprise web application is based on the three-tier
framework: the server layer, the application layer, and the data layer (Sakr & Gaber, 2014).
The data partitioning and the data replication are two commonly used
approaches to achieve the availability, scalability, and performance
enhancement goals in the distributed data management. There are two main approaches to achieve
scalability at the database layer to be able to handle the client requests when
the load of the application is increased (Sakr & Gaber, 2014).
The first approach is to scale up allocating a bigger machine to act as
database servers. The second approach is to scale out replicating and
partitioning data across more machines (Sakr & Gaber, 2014). However, the traditional database
suffers from serious limitations. Database
systems are not easy to scale as they cannot exceed a certain limit. The database systems are not easy to configure and
maintain. The specialized database
systems for main memory systems as the case with OLTP and column stores as the
case with OLAP add more complication and cost when selecting the database
system in addition to the peak provisioning unnecessary cost (Sakr & Gaber, 2014).
Thus, new systems called NoSQL started to
emerge as an alternative mode for the traditional database systems to be able
to deal and handle Big Data (Moniruzzaman & Hossain, 2013; Pokorny, 2013; Sakr
& Gaber, 2014). Not Only SQL” known as NoSQL databases
emerged to deal with Big Data. NoSQL
database systems were developed by major
internet companies such as Facebook, Amazon, and Google, when they were confronted with the Big Data challenges (Moniruzzaman & Hossain, 2013).
NoSQL databases are found to be suitable for massive scheme-free
datasets for Big Data management (Hu, Wen, Chua, & Li, 2014).
NoSQL database systems are considered to be the potential data management
solution for Big Data (Abbasi, Sarker, & Chiang, 2016).
NoSQL
Data Storage and the Tradeoff between Consistency and Availability
The ACID properties are regarded to be one
of the basic features of the traditional relational databases (Moniruzzaman & Hossain, 2013; Pokorny, 2013; Sakr
& Gaber, 2014). ACID stands for “Atomicity,” “Consistency,” “Isolation,” and “Durability” (Pokorny, 2013).
These ACID properties indicate “all or nothing” concept behind the
traditional database (Pokorny, 2013).
The relational database has been full
compliance with ACID principle (Pokorny, 2013).
In addition to the ACID properties, there is also CAP theorem which
states that for any system sharing data it is impossible to guarantee all of these three properties simultaneously (Pokorny,
2013). These three properties of
CAP include “consistency,” “availability,” and “partition tolerance” (Pokorny, 2013; Sakr & Gaber, 2014).
Moreover, the traditional relational database is also characterized by a schema, where data is structured in tables, tuples, and fields (Moniruzzaman & Hossain, 2013; Sadalage &
Fowler, 2013). The traditional consistency model is not
adequate for distributed systems such as the Cloud environment (Sakr & Gaber, 2014).
There are two major consistency models;
the strong consistency which includes the linearizability and serializability,
and weak consistency which includes the eventual the causal, eventual, and
timeline consistency model (Sakr & Gaber, 2014).
The causal consistency model ensures total ordering between operations
which have causal relations. The eventual consistency model ensures all
replicas will gradually and eventually become consistent in the absence of
updates (Sakr & Gaber, 2014). The timeline consistency model
guarantees all replicas perform operations on one record in the same “correct
order” (Sakr & Gaber, 2014).
As indicated in (Pokorny, 2013), the NoSQL database systems scale
nearly linearly with the number of servers used (Pokorny, 2013). The reason for such capability
for scaling nearly linearly is due to the use of “data partitioning” (Pokorny, 2013).
In NoSQL database systems, the method of distributed hash tables (DHT) is often used, in which couples of (key, value)
are hashed into buckets – partial storage
spaces, each from that placed in a node (Pokorny, 2013). NoSQL is not characterized by a schema or structured data (Hu et al., 2014; Sadalage & Fowler, 2013).
NoSQL systems are fast, highly scalable, and reliable (Hu et al., 2014). The term of “NoSQL” database
indicates the loosely specified class of non-relational data stores (Pokorny, 2013).
NoSQL databases mostly do not use SQL as their query language (Pokorny, 2013).
They do not support operations of “JOIN” and “ORDER BY.” The
reason is that because portioning data is done
horizontally (Pokorny, 2013).
The data in NoSQL is often organized into tables on a logical level and
accessed only through the Primary Key (Pokorny, 2013).
NoSQL database systems are organized by data models (Hu et al., 2014). They are categorized as (1) key-value stores, (2) column-oriented databases, and (3) document databases (Hu et al., 2014). Most NoSQL database systems are key-value stores or big hash tables, which contain a pair of the key-value dataset (Pokorny, 2013). This approach of the key-value or big hash table increases the efficiency of the lookup (Pokorny, 2013). The key uniquely identifies a value typically a string but it can also be a pointer, where the value is stored. The value can be structured or completely unstructured typically in BLOB (binary large object) format (Pokorny, 2013). The pair of key-value can be of different types, and may not come from the same table (Pokorny, 2013). These characteristics of the NoSQL database using the pair of key-value concept, increase efficiency and scalability of the NoSQL database systems (Pokorny, 2013). Column-oriented databases store and process data by column instead of by row (Hu et al., 2014). The rows and columns will get split over multiple nodes to achieve scalability (Hu et al., 2014). The document-oriented databases support more complex data structures than the key-value stores (Hu et al., 2014). The document-oriented database systems are the most general models of the NoSQL databases (Pokorny, 2013). There are other data models for NoSQL dataset systems, including the graph databases that are not discussed in this paper. Table 1 illustrates some of these data models of the NoSQL databases reflecting the name of the databases for each model, producer, CAP option, and consistency, derived from (Hu et al., 2014).
Table 1: NoSQL
Database Models. Adapted from (Hu et al., 2014)
In accordance (Hu et al., 2014; Moniruzzaman & Hossain, 2013; Pokorny, 2013), some of these NoSQL databases do not implement ACID fully, and can only be eventually consistent (Hu et al., 2014; Moniruzzaman & Hossain, 2013; Pokorny, 2013). NoSQL databases implement “eventual consistency” concept instead of “strong consistency,” where any updates and changes are replicated to the entire database eventually, but at any given time (Hu et al., 2014; Moniruzzaman & Hossain, 2013; Pokorny, 2013). The “eventually consistent” term means that the system will eventually become consistent but after some time (Hu et al., 2014; Moniruzzaman & Hossain, 2013; Pokorny, 2013). This principle of “eventual consistency” indicate more availability and greatly improve scalability at the expense of the full and immediate consistency (Hu et al., 2014; Moniruzzaman & Hossain, 2013; Pokorny, 2013). NoSQL database has particular architectures that use different possibilities of distribution, ensuring the availability of the data, and the access to the data using replication techniques. Figure 1 illustrates the characteristics of NoSQL databases, derived from (Moniruzzaman & Hossain, 2013).
Figure 1: Characteristics of NoSQL databases. Adapted from
(Moniruzzaman &
Hossain, 2013)
In accordance to (Chen & Zhang, 2014), the most popular NoSQL database
is Apache Cassandra. Facebook in 2008
released Cassandra as open source (Chen & Zhang, 2014).
Other NoSQL implementations include Apache Hadoop, MapReduce,
MemcacheDB, and Voldemort (Chen & Zhang, 2014).
References
Abbasi, A., Sarker, S., &
Chiang, R. (2016). Big data research in information systems: Toward an
inclusive research agenda. Journal of the
Association for Information Systems, 17(2), 3.
Chen, C. P.,
& Zhang, C.-Y. (2014). Data-intensive applications, challenges, techniques,
and technologies: A survey of Big Data. Information
Sciences, 275, 314-347.
Hu, H., Wen, Y.,
Chua, T.-S., & Li, X. (2014). Toward scalable systems for big data
analytics: A technology tutorial. IEEE
Access, 2, 652-687.
Moniruzzaman, A.,
& Hossain, S. A. (2013). NoSQL database: New era of databases for big data
analytics-classification, characteristics, and comparison. arXiv preprint arXiv:1307.0191.
Pokorny, J.
(2013). NoSQL databases: a step to database scalability in a web environment. International Journal of Web Information
Systems, 9(1), 69-82.
Sadalage, J. P.,
& Fowler, M. (2013). NoSQL: A Brief
Guide to the Emerging World of Polyglot Persistence (1st Edition ed.):
Addison-Wesley.
Sakr,
S., & Gaber, M. (2014). Large Scale
and big data: Processing and Management: CRC Press.
The traditional distributed models involve the Shared Memory and the Message Passing programming models. The SM and the MP techniques are two major parallel computing architecture models (Coulouris, Dollimore, & Kindberg, 2005; Manoj, Manjunath, & Govindarajan, 2004). They provide the basic interaction model for distributed tasks and lack any facility to automatically parallelize and distribute tasks or tolerate faults (Hammoud, Rehman, & Sakr, 2012; Sakr & Gaber, 2014). Advanced models such as MapReduce, Pregel, and GraphLab are introduced to overcome the inefficiencies and challenges of these traditional distributed models especially when porting them to the Cloud environment (Fernández et al., 2014; Low et al., 2012; Sakr & Gaber, 2014). These new models are built upon these two traditional models of the SM and MP and offer various key properties for the Cloud environment (Sakr & Gaber, 2014). These models are also referred to as “distributed analytics engines” (Sakr & Gaber, 2014).
Shared Memory Distributed Programming
The SM system has a global memory that is accessible to all the processors in the system (Manoj et al., 2004). The SM techniques include two models based on the nature of sharing of this global memory across processors. The first model is the Uniform-Memory-Access (UMA) (Manoj et al., 2004; Pulipaka, 2016). The second model is the Non-Uniform-Memory-Access (NUMA) (Coulouris et al., 2005; Dean & Ghemawat, 2004; Hennessy & Patterson, 2011; Manoj et al., 2004; Mishra, Dehuri, & Kim, 2016). The access time to the memory in UMA model from two processors is equal. The access time to the memory in NUMA varies for different processors (Manoj et al., 2004). Hardware Distributed Shared Memory (DSM) is an example of the NUMA including Stanford DASH, SCI, DDM, and KSRI (Manoj et al., 2004). When using a uniprocessor environment, the programming in SM systems involves updating shared data, which is regarded to be a simple programming process (Manoj et al., 2004). However, with the increasing number of processors, the environment experiences increased contention and long latencies in accessing the shared memory (Manoj et al., 2004). The contention and latencies diminish the performance and limit scalability (Manoj et al., 2004). Additional issues with the SM systems involve data access synchronization, cache coherency, and memory consistency (Manoj et al., 2004). The developers must ensure appropriate memory access order through synchronization primitives (Manoj et al., 2004). Moreover, when using hardware in the implementation of the SM abstraction, the cost of the system gets increased (Manoj et al., 2004).
In SM programming model, the
data is not explicitly communicated but implicitly exchanged via sharing which
entails the use of synchronization
techniques within the distributed programs (Sakr & Gaber, 2014). The tasks can
communicate by reading and writing to the shared memory or disks
locations. The tasks can access any
location in the distributed memories/disk which is similar to the threads of a
single process in operating systems, where all threads share the process
address space and communicate by reading and writing to that space (Sakr & Gaber, 2014). The distributed tasks are prevented from simultaneously writing to a
shared data to avoid inconsistent or corrupted data
when using synchronization which is
required to control the order in which read/write operations are performed by various tasks (Sakr & Gaber, 2014). The communication
between two tasks is implicit through reads and writes to shared arrays and
variables and synchronization is explicit through locks and barriers (Sakr & Gaber, 2014).
The SM programming model must meet two main
requirements: developers do not need to
explicitly encode functions to send/receive messages, and the underlying
storage layer provides a shared view of
all tasks. MapReduce satisfies these two
requirements using a map and reduce, and the communications occur only between the
map and reduce phases under the full control of the MapReduce engine. Moreover,
the synchronization is also handled by
the MapReduce engine (Sakr & Gaber, 2014). For the storage layer,
MapReduce utilizes the Hadoop Distributed File System (HDFS) which provides a
shared abstraction for all tasks, where any task transparently access any location in HDFS (Sakr & Gaber, 2014). Thus, MapReduce
provides the shared-memory abstraction which is provided internally by Hadoop
that entails MapReduce engine and HDSF (Sakr & Gaber, 2014). GraphLab also offers a
shared-memory abstraction by eliminating the need for users to send/receive messages in update functions
explicitly and provides a shared view among vertices in a graph (Fernández et al., 2014; Low et
al., 2012; Sakr & Gaber, 2014). GraphLab allows scopes of vertices to overlap
and vertices to read and write from and to their scopes which can cause a conflict of read-write and write-write between
vertices sharing scope (Fernández et al., 2014; Low et
al., 2012; Sakr & Gaber, 2014).
Message Passing Distributed Programming
In MP programming model the distributed tasks communicate by sending and receiving messages where the distributed tasks do not share an address space at which they can access each other’s memories (Sakr & Gaber, 2014). The MP programming model provides the abstraction which is similar to that of the process and not the threads in the operating system (Sakr & Gaber, 2014). This model incurs communications overheads for explicitly sending and receiving messages that contain data such as variable network latency, potentially excessive data transfer (Hammoud et al., 2012; Sakr & Gaber, 2014). When using MP programming model, no explicit synchronization is needed because for every send operation; there is a corresponding receive operation (Sakr & Gaber, 2014). Moreover, no illusion for single shared address space is required for the distributed system for tasks to interact (Sakr & Gaber, 2014).
Message Passing Interface (MPI)
is a popular example of the MP programming model, which is an industry-standard library for writing
message-passing programs (Hammoud et al., 2012; Sakr &
Gaber, 2014). Pregel is regarded the common analytics
engine which utilizes the message-passing programming model (Malewicz et al., 2010; Sakr
& Gaber, 2014). In
Pregel, vertices can communicate only by sending and receiving messages, which
should be explicitly encoded (Malewicz et al., 2010; Sakr
& Gaber, 2014). The SM
programs are easier to develop than the MP programs (Manoj et al., 2004; Sakr &
Gaber, 2014). The code structure of SM programs is often
not much different than its respective sequential one, with only additional
directives to be added to specify parallel/distributed tasks, the scope of variables, and synchronization points (Manoj et al., 2004; Sakr &
Gaber, 2014). Large-scale distributed systems such as the
Cloud imply non-uniform access latencies such as accessing remote data takes
more time than accessing local data, thus enforcing programmers to lay out data
close to relevant tasks (Sakr & Gaber, 2014).
References
Coulouris, G. F., Dollimore, J.,
& Kindberg, T. (2005). Distributed
systems: concepts and design: pearson education.
Dean, J., &
Ghemawat, S. (2004). MapReduce: simplified data processing on large clusters.
OSDI’04 Proceedings of the 6th conference on Symposium on Opearting Systems
Design and Implementation” dalam: International Journal of Engineering Science
Invention. 10-100.
Fernández, A.,
del Río, S., López, V., Bawakid, A., del Jesus, M. J., Benítez, J. M., &
Herrera, F. (2014). Big Data with Cloud Computing: an insight on the computing
environment, MapReduce, and programming frameworks. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 4(5),
380-409.
Hammoud, M., Rehman,
M. S., & Sakr, M. F. (2012). Center-of-gravity
reduce task scheduling to lower mapreduce network traffic. Paper presented
at the Cloud Computing (CLOUD), 2012 IEEE 5th International Conference on.
Hennessy, J. L.,
& Patterson, D. A. (2011). Computer
architecture: a quantitative approach: Elsevier.
Low, Y., Bickson,
D., Gonzalez, J., Guestrin, C., Kyrola, A., & Hellerstein, J. M. (2012).
Distributed GraphLab: a framework for machine learning and data mining in the
cloud. Proceedings of the VLDB Endowment,
5(8), 716-727.
Malewicz, G.,
Austern, M. H., Bik, A. J., Dehnert, J. C., Horn, I., Leiser, N., &
Czajkowski, G. (2010). Pregel: a system
for large-scale graph processing. Paper presented at the Proceedings of the
2010 ACM SIGMOD International Conference on Management of data.
Manoj, N.,
Manjunath, K., & Govindarajan, R. (2004). CAS-DSM: A compiler assisted
software distributed shared memory. International
Journal of Parallel Programming, 32(2), 77-122.
Mishra, B. S. P.,
Dehuri, S., & Kim, E. (2016). Techniques
and Environments for Big Data Analysis: Parallel, Cloud, and Grid Computing
(Vol. 17): Springer.
MapReduce and RDF Data
Query Processing Optimized Performance
This project
provides a survey on the state-of-the-art techniques for applying MapReduce to
improve the Resource Description Framework (RDF) data query processing
performance. There has been a tremendous
effort from the industry and researchers to develop efficient and scalable RDF
processing systems. Most of the complex
data-processing tasks require multiple cycles of the MapReduce which are
chained together into sequential. The decomposition
of a task into cycles or subtasks are often implemented. Thus, the low overall workflow cost is a key
element in the decomposition. Each cycle
of the MapReduce results in significant overhead. When using RDF, the decomposition problem
reflects the distribution of operations such as SELECT, and JOIN into subtasks which
is supported by MapReduce cycle. The issue of the decomposition is related to
the operations order because the neighboring operation in a query plan can be
effectively grouped into the same subtasks.
When using MapReduce, the operation order is based on the requirement of
key partitioning so that the neighboring operations do not cause any conflict. Various techniques are proposed to enhance
the performance of the semantic web queries using RDF and MapReduce.
This
project begins with the overview of RDF, followed by RDF Store Architecture,
MapReduce Parallel Processing Framework, and Hadoop. RDF and SPARQL using semantic query is
discussed covering the syntax of SPARQL and the missing features that are
required to enhance the performance of RDF using MapReduce. Various techniques are discussed and analyzed
on the application of MapReduce to improve RDF query processing
performance. Some of these techniques
include HadoopRDF, RDFPath, and PigSPARQL.
Resource Description
Framework (RDF)
Resource
Description Framework (RDF) is described as an emerging standard for processing
metadata (Punnoose,
Crainiceanu, & Rapp, 2012; Tiwana & Balasubramaniam, 2001) (Punnoose
et al., 2012; Tiwana & Balasubramaniam, 2001). RDF provides interoperability between
applications that exchange machine-understandable information on the Web (Sakr
& Gaber, 2014; Tiwana & Balasubramaniam, 2001). The primary goal of RDF is to define a
mechanism and provide standards for the metadata and for describing resources on
the Web (Boussaid,
Tanasescu, Bentayeb, & Darmont, 2007; Firat & Kuzu, 2011; Tiwana &
Balasubramaniam, 2001) which makes no a priori assumptions
about a particular application domain or the associated semantics (Tiwana
& Balasubramaniam, 2001). These standards or mechanisms provided by the
RDF can prevent users from accessing irrelevant subjects because RDF provided
metadata that is relevant to the desired information (Firat
& Kuzu, 2011).
RDF
is also described as a Data Model (Choi,
Son, Cho, Sung, & Chung, 2009; Myung, Yeon, & Lee, 2010) for representing
labeled directed graphs (Choi
et al., 2009; Nicolaidis & Iniewski, 2017), and useful for a
Data Warehousing solution as the MapReduce framework (Myung
et al., 2010).
RDF is used as an important building block of Semantic Web of Web 3.0
(see Figure 1) (Choi
et al., 2009; Firat & Kuzu, 2011). The technologies of the Semantic Web are
useful for maintaining data in the Cloud (M.
F. Husain, Khan, Kantarcioglu, & Thuraisingham, 2010). These technologies of the Semantic Web
provide the ability to specify and query heterogeneous data in a standard
manner (M.
F. Husain et al., 2010).
RDF
Data Model can be extended to ontologies to include RDF Schema (RDFS) and Ontology
Web Language (OWL) to provide techniques to define and identify vocabularies
specified to a certain domain, schema and relations between the elements of the
vocabulary (Choi
et al., 2009).
RDS can be exported in various
file formats (Sun
& Jara, 2014).
The most common of these formats is RDF + XML and XSD (Sun
& Jara, 2014).
The OWL is used to add semantics to the schema (Sun
& Jara, 2014).
For instance, if “A isAssociatedWith B,” which implies that “B is AssociatedWith
A” (Sun
& Jara, 2014).
The OWL allows the ability to express these two things the same way (Sun
& Jara, 2014).
This similarity feature allowed by OWL is very useful for “joining” data
expressed in different schemas (Sun
& Jara, 2014).
This feature allows building relationship and joining up data from
multiple sites, described as “Linked Data” facilitating the heterogeneous data
stream integration (Sun
& Jara, 2014).
OWL enables new facts to be derived from known facts using the inference
rules (Nicolaidis
& Iniewski, 2017).
Another example which can be used to enforce the inference technique
using OWL is when a triple states that a car is a subtype of a vehicle and
another triple state that a Cabrio is a subtype of a car, the new fact will be
that Cabrio is a vehicle which can be inferred from the previous facts (Nicolaidis
& Iniewski, 2017).
RDF Data Model is described to be a simple and flexible framework (Myung et al., 2010). The underlying form and expression in RDF is a collection of “triples,” each consisting of a subject (s), a predicate (p), and an object (o) (Brickley, 2014; Connolly & Begg, 2015; Nicolaidis & Iniewski, 2017; Przyjaciel-Zablocki, Schätzle, Skaley, Hornung, & Lausen, 2013; Punnoose et al., 2012). The subjects and predicates are Resources; each encoded as a Uniform Resource Identifier (URI) to ensure the uniqueness, while the object can be a Resource or a Literal such as string, date or number (Nicolaidis & Iniewski, 2017). In (Firat & Kuzu, 2011), the basic structure of the RDF Data Model is based on a triplet of the object (O), quality (A) and value (V) (Firat & Kuzu, 2011). The basic role of RDF is to provide Data Model of the object, quality and value (OAV) (Firat & Kuzu, 2011). RDF Data Model is similar to the XML Data Model, where both do not include form-related information or names (Firat & Kuzu, 2011).
Figure 1: The Major Building Blocks of the Semantic Web Architectures. Adapted from (Firat & Kuzu, 2011).
RDF has been commonly used in
applications such as Semantic Web, Bioinformatics, and Social Networks because
of its great flexibility and applicability (Choi
et al., 2009).
These applications require a huge computation over a large set of data (Choi
et al., 2009).
Thus, the large-scale graph datasets are very common among these
applications of Semantic Web, Bioinformatics, and Social Networks (Choi
et al., 2009).
However, the traditional techniques for processing such large-scale of
the dataset are found to be inadequate (Choi
et al., 2009).
Moreover, RDF Data Model enables
existing heterogeneous database systems to be integrated into a Data Warehouse
because of its flexibility (Myung
et al., 2010).
The flexibility of the RDF Data Model also provides users the inference
capability to discover unknown knowledge which is useful for large-scale data
analysis (Myung
et al., 2010). RDF triples require terabytes of disk space
for storage and analysis (M.
Husain, McGlothlin, Masud, Khan, & Thuraisingham, 2011; M. F. Husain et
al., 2010).
Researchers are encouraged to develop efficient repositories, because
there are only a few existing frameworks such as RDF-3X, Jena, Sesame, BigOWLIM
for Semantic Web technologies (M.
Husain et al., 2011; M. F. Husain et al., 2010).
These frameworks are single-machine RDF systems and are widely used
because they are user-friendly and perform well for small and medium-sized RDF datasets
(M.
Husain et al., 2011; M. F. Husain et al., 2010; Sakr & Gaber, 2014).
The RDF-3X is regarded to be the fastest single machine RDF systems regarding
query performance that vastly outperforms previous single machine systems (M.
Husain et al., 2011; M. F. Husain et al., 2010; Sakr & Gaber, 2014).
However, the performance of RDF-3X diminishes for queries with unbound
objects and low selectivity factor (M.
Husain et al., 2011; M. F. Husain et al., 2010; Sakr & Gaber, 2014).
These frameworks are confronted by the large RDF graphs (M.
Husain et al., 2011; M. F. Husain et al., 2010). Therefore, the storage of a large volume of
RDF triples and the efficient query of the RDF triples are challenging and are regarded
to be critical problems in Semantic Web (M.
Husain et al., 2011; M. F. Husain et al., 2010; Sakr & Gaber, 2014).
These challenges also limit the scaling capabilities (M.
Husain et al., 2011; M. F. Husain et al., 2010; Sakr & Gaber, 2014).
RDF Store Architecture
The main purpose of the RDF store is to build a database for storing and retrieving data of any data expressed in RDF (Modoni, Sacco, & Terkaj, 2014). The term RDF store is used as an abstract for any system that can handle RDF data, allowing the ingestion of serialized RDF data and the retrieval of these data, and providing a set of APIs to facilitate the integration with other third-party application as the client application (Modoni et al., 2014). The term triple store often refers to these types of systems (Modoni et al., 2014). RDF store includes two major components; the Repository and the Middleware. The Repository represents a set of files or database (Modoni et al., 2014). The Middleware is on top of the repository and in constant communication with it (Modoni et al., 2014). Figure 2 illustrates the RDF store architecture. The Middleware has its components; Storage Provider, Query Engine, Parser/Serializer, and Client Connector (Modoni et al., 2014). The current RDF stores are categorized into three groups; database based stores, native stores, and hybrid stores (Modoni et al., 2014). Examples of the database based stores include MySQL, Oracle 12c, which are built on top of existing commercial database engines (Modoni et al., 2014). Examples of native stores are AllegroGraph, OWLIM which are built as database engines from scratch (Modoni et al., 2014). Examples of the hybrid stores are Virtuoso, and Sesame which supports architectural styles; native and DBMS-backed (Modoni et al., 2014).
Figure 2: RDF
Store Architecture (Modoni et al., 2014)
MapReduce Parallel
Processing Framework and Hadoop
In 2004, Google introduced MapReduce framework as a Parallel Processing
framework which deals with a large set of
data (Bakshi, 2012; Fadzil, Khalid, &
Manaf, 2012; White, 2012-important-buildingblocks). The MapReduce framework has gained much
popularity because it has features for hiding sophisticated operations of the
parallel processing (Fadzil et al., 2012). Various MapReduce frameworks
such as Hadoop were introduced because of the enthusiasm towards MapReduce (Fadzil et al., 2012). The capability of the MapReduce
framework was realized by different research areas such as data warehousing,
data mining, and the bioinformatics (Fadzil et al., 2012). MapReduce framework consists of
two main layers; the Distributed File System (DFS) layer to store data and the
MapReduce layer for data processing (Lee, Lee, Choi, Chung, & Moon, 2012;
Mishra, Dehuri, & Kim, 2016; Sakr & Gaber, 2014). DFS is a major feature of the MapReduce
framework (Fadzil et al., 2012).
MapReduce framework is using large clusters of low-cost
commodity hardware to lower the cost (Bakshi, 2012; H. Hu, Wen, Chua, & Li, 2014;
Inukollu, Arsi, & Ravuri, 2014; Khan et al., 2014; Krishnan, 2013; Mishra
et al., 2016; Sakr & Gaber, 2014; White, 2012-important-buildingblocks). MapReduce framework is using
“Redundant Arrays of Independent (and inexpensive) Nodes (RAIN),” whose
components are loosely coupled and when
any node goes down, there is no negative
impact on the MapReduce job (Sakr & Gaber, 2014; Yang, Dasdan,
Hsiao, & Parker, 2007). MapReduce framework involves the
“Fault-Tolerance” by applying the replication technique and allows replacing
any crashed nodes with another node without affecting the currently running job (P. Hu & Dai, 2014; Sakr & Gaber,
2014). MapReduce framework involves the automatic
support for the parallelization of execution which makes the MapReduce highly
parallel and yet abstracted (P. Hu & Dai, 2014; Sakr & Gaber,
2014).
MapReduce was introduced to solve the
problem of parallel processing of a large set
of data in a distributed environment which required manual management of the
hardware resources (Fadzil et al., 2012; Sakr & Gaber,
2014). The complexity of the parallelization is solved by using two techniques: Map/Reduce technique, and Distributed File
System (DFS) technique (Fadzil et al., 2012; Sakr & Gaber,
2014). The parallel framework must be reliable to
ensure good resource management in the distributed environment using
off-the-shelf hardware to solve the scalability issue to support any future
requirement for processing (Fadzil et al., 2012). The earlier frameworks such as
the Message Passing Interface (MPI) framework was having a reliability issue
and had a fault-tolerance issue when
processing a large set of data (Fadzil et al., 2012). MapReduce framework covers the
two categories of the scalability; the structural scalability, and the load
scalability (Fadzil et al., 2012). It addresses the structural
scalability by using the DFS which allows forming large virtual storage for the
framework by adding off-the-shelf hardware.
MapReduce framework addresses the load scalability by increasing the
number of the nodes to improve the performance (Fadzil et al., 2012).
However,
the earlier version of MapReduce framework faced challenges. Among these
challenges are the join operation and the
lack of support for aggregate functions to join multiple datasets in one task (Sakr
& Gaber, 2014).
Another limitation of the standard MapReduce framework is found in the iterative processing which is required for analysis techniques such as
PageRank algorithm, recursive relational queries, and social network analysis (Sakr
& Gaber, 2014).
The standard MapReduce does not share the execution of work to reduce
the overall amount of work (Sakr
& Gaber, 2014).
Another limitation was found in the lack of support of data index and
column storage but support only for a sequential
method when scanning the input data. Such a lack of data index affected
the query performance (Sakr
& Gaber, 2014).
Moreover, many argued that MapReduce is not regarded to be the optimal
solution for structured data. It is
known as shared-nothing architecture, which supports scalability (Bakshi,
2012; Jinquan, Jie, Shengsheng, Yan, & Yuanhao, 2012; Sakr & Gaber,
2014; White, 2012-important-buildingblocks), and the processing of large
unstructured data sets (Bakshi,
2012).
MapReduce has the limitation of performance and efficiency (Lee
et al., 2012).
Hadoop is a software framework which is derived from Big
Table and MapReduce and managed by Apache.
It was created by Doug Cutting and was named after his son’s toy
elephant (Mishra et al.,
2016). Hadoop allows applications to run on huge
clusters of commodity hardware based on MapReduce (Mishra et al.,
2016). The underlying concept of Hadoop is to allow
the parallel processing of the data across different computing nodes to speed
up computations and hide the latency (Mishra et al.,
2016). The Hadoop Distributed File System (HDFS) is
one of the major components of the Hadoop framework for storing large files (Bao, Ren, Zhang,
Zhang, & Luo, 2012; Cloud Security Alliance, 2013; De Mauro, Greco, &
Grimaldi, 2015) and allowing
access to data scattered over multiple nodes in without any exposure to the
complexity of the environment (Bao et al., 2012;
De Mauro et al., 2015). The MapReduce programming model is another major component
of the Hadoop framework (Bao
et al., 2012; Cloud Security Alliance, 2013; De Mauro et al., 2015) which is designed to implement the
distributed and parallel algorithms efficiently (De
Mauro et al., 2015).
HBase is the third component of Hadoop framework (Bao
et al., 2012).
HBase is developed on the HDFS and is a NoSQL (Not only SQL) type
database (Bao
et al., 2012).
The
key features of Hadoop include the scalability and flexibility, cost efficiency
and fault tolerance (H.
Hu et al., 2014; Khan et al., 2014; Mishra et al., 2016; Polato, Ré, Goldman,
& Kon, 2014; Sakr & Gaber, 2014).
Hadoop allows the nodes in the cluster to scale up and down based on the
computation requirements and with no change in the data formats (H.
Hu et al., 2014; Polato et al., 2014).
Hadoop also provides massively parallel computation to commodity
hardware decreasing the cost per terabyte of storage which makes the massively
parallel computation affordable when the volume of the data gets increased (H.
Hu et al., 2014).
The Hadoop technology offers the flexibility feature as it is not tight
with a schema which allows the utilization of any data either structured,
non-structures, and semi-structured, and
the aggregation of the data from multiple sources (H.
Hu et al., 2014; Polato et al., 2014).
Hadoop also allows nodes to crash without affecting the data
processing. It provides fault tolerance
environment where data and computation can be recovered without any negative
impact on the processing of the data (H.
Hu et al., 2014; Polato et al., 2014; White, 2012-important-buildingblocks).
RDF and SPARQL Using
Semantic Query
Researchers
have exerted effort in developing Semantic Web technologies which have been standardized
to address the inadequacy of the current traditional analytical techniques (M.
Husain et al., 2011; M. F. Husain et al., 2010). RDF, SPARQL
(Simple Protocol And RDF Query Language) are the most prominent standardized semantic
web technologies (M.
Husain et al., 2011; M. F. Husain et al., 2010). The Data Access Working Group (DAWG) of the
World Wide Web Consortium (W3C) in 2007 recommended SPARQL and provided
standards to be the query language for RDF, a protocol definition for sending
SPARQL queries from a client to a query processor and an XML-based serialization
format for results returned by the SPARQL query (Konstantinou,
Spanos, Stavrou, & Mitrou, 2010; Sakr & Gaber, 2014). RDF is regarded to be the standard for
storing and representing data (M.
Husain et al., 2011; M. F. Husain et al., 2010). SPARQL is the query language to retrieve data
from RDF triplestore (M.
Husain et al., 2011; M. F. Husain et al., 2010; Nicolaidis & Iniewski,
2017; Sakr & Gaber, 2014; Zeng, Yang, Wang, Shao, & Wang, 2013).
Like RDF, SPARQL is built on the “triple pattern,” which also contains
the “subject,” “predicate,” and “object” and is terminated with a full stop (Connolly
& Begg, 2015).
RDF triple is regarded to be a SPARQL triple pattern (Connolly
& Begg, 2015).
URIs are written inside angle brackets for identifying resources;
literal strings are denoted with either double or single quote; properties,
like Name, can be identified by their URI or more normally using a QName-style
syntax to improve readability (Connolly
& Begg, 2015). The triple pattern can include
variables which are not like the triple (Connolly
& Begg, 2015). Any or all of the values of
subject, predicate, and object in a triple pattern may be replaced by a
variable, which indicates data items of interest that will be returned by a
query (Connolly
& Begg, 2015).
The semantic query plays a significant role in the Semantic Web, and the standardization of SPARQL plays a significant role to achieve such semantic queries (Konstantinou et al., 2010). Unlike the traditional query languages, SPARQL does not consider the graph level, but rather it models the graph as a set of triples (Konstantinou et al., 2010). Thus, when using the SPARQL query, the graph pattern is identified, and the nodes which match this pattern are returned (Konstantinou et al., 2010; Zeng et al., 2013). SPARQL syntax is similar to SQL such as SELECT FROM WHERE syntax which is the most striking syntax (Konstantinou et al., 2010). The core syntax of SPARQL is a conjunctive set of triple patterns called as “basic graph pattern” (Zeng et al., 2013). Table 1 shows the syntax of SPARQL to retrieve data from RDF using SELECT statement.
Table 1: Example
of SPARQL syntax (Sakr & Gaber, 2014)
Although SPARQL syntax is similar to SQL
in the context of SELECT to retrieve data, SPARQL is not as mature as SQL (Konstantinou
et al., 2010).
The current form of SPARQL allows the access to the raw data using URIs
from RDF or OWL graph and letting the user perform the result processing (Konstantinou
et al., 2010).
However, SPARQL is expected to be the gateway to query information and
knowledge supporting as many features as SQL does (Konstantinou
et al., 2010).
SPARQL does not support any aggregated functions such as MAX, MIN, SUM,
AVG, COUNT, and the GROUP BY operations (Konstantinou
et al., 2010).
Moreover, SPARQL supports ORDER BY only on a global level and not solely
on the OPTIONAL part of the query (Konstantinou
et al., 2010).
For mathematical operations, SPARQL does not extend its support beyond
the basic mathematical operations (Konstantinou
et al., 2010).
SPARQL does not support the nested queries, meaning it does not allow
CONSTRUCT query in the FROM part of the query.
Moreover, SPARQL is missing the functionality offered by SELECT WHERE
LIKE statement in SQL, allowing for keyword-based queries (Konstantinou
et al., 2010).
While SPARQL offers regex() function for string pattern matching, it
cannot emulate the functionality of the LIKE operator (Konstantinou
et al., 2010).
SPARQL enables only the unbound variables in the SELECT part and
rejecting the use of functions or other operators (Konstantinou
et al., 2010). This limitation places SPARQL as
elementary query language where URIs or literals only are returned, while users
look for some result processing in the practical use cases (Konstantinou
et al., 2010).
SPARQL can be enhanced to include these missing features and
functionality to include stored procedures, triggers, and operations for data
manipulations such as update, insert, and delete (Konstantinou
et al., 2010).
There
is a group called SPARQL Working Group who are working on integrating these missing
features in SPARQL. SPARQL/Update is an
extension to SPARQL included in the leading Semantic Web development framework
“Jena” allowing the update operation, the creation and the removal of the RDF
graphs (Konstantinou
et al., 2010). ARQ is a query engine for Jena
which supports the SPARQL RDF Query language (Apache,
2017a). Some of the key features of the ARQ include
the update, the GROUP BY, access and extension of the SPARQL algebra, and
support for the federated query (Apache,
2017a). LARQ integrates SPARQL with Apache’s
full-text search framework Lucene (Konstantinou
et al., 2010) adding free text searches to SPARQL (Apache,
2017b). SPARQL+ extension of the ARC RDF sore offers
most of the common aggregates and extends the SPARUL’s INSERT with CONSTRUCT
clause (Konstantinou
et al., 2010).
The OpenLink’s Virtuoso extends SPARQL with aggregate functions, nesting,
and subqueries, allowing the user to insert SPARQL queries inside SQL (Konstantinou
et al., 2010).
SPASQL offers a similar functionality embedding SPARQL into SQL (Konstantinou
et al., 2010).
Although
SPARQL is missing a lot of SQL features, it offers other features which are not
part of SQL (Konstantinou
et al., 2010).
Some of these features include the OPTIONAL operator which does not
modify the results in case of non-existence and it can be met in almost all of
the query languages for RDF (Konstantinou
et al., 2010).
This feature is equivalent to the LEFT OUTER JOIN in SQL (Konstantinou
et al., 2010).
However, SPARQL syntax is much more user-friendly and intuitive than SQL
(Konstantinou
et al., 2010).
Techniques Applied on MapReduce
To Improve RDF Query Processing
Performance
With
the explosive growth of the data size, the traditional approach of analyzing
the data in a centralized server is not adequate to scale up (Punnoose
et al., 2012; Sakr & Gaber, 2014), and cannot scale
concerning the increasing RDF datasets (Sakr
& Gaber, 2014).
Although SPARQL is used to query RDF data, the query of RDF dataset at
the web scale is challenging because the computation of SPARQL queries requires
several joins between subsets of the dataset (Sakr
& Gaber, 2014).
New methods are introduced to improve the parallel computing and allow
storage and retrieval of RDF across large compute clusters which enables
processing data of unprecedented magnitude (Punnoose
et al., 2012).
Various solutions are introduced to solve these challenges and achieve
scalable RDF processing using the MapReduce framework such as PigSPARQL, and
RDFPath.
RDFPath
In (Przyjaciel-Zablocki, Schätzle, Hornung, & Lausen, 2011), the RDFPath is proposed as a declarative path query language for RDF which provides a natural mapping to the MapReduce programming model by design, while remaining extensible (Przyjaciel-Zablocki et al., 2011). It supports the exploration of graph properties such as shortest connections between two nodes in an RDF graph (Przyjaciel-Zablocki et al., 2011). RDFPath is regarded to be a valuable tool for the analysis of social graphs (Przyjaciel-Zablocki et al., 2011). RDFPath combines an intuitive syntax for path queries with an effective execution strategy using MapReduce (Przyjaciel-Zablocki et al., 2011). RDFPath does benefit from the horizontal scaling properties of MapReduce when adding more nodes to improve the overall executions time significantly (Przyjaciel-Zablocki et al., 2011). Using RDFPath, large RDF graphs can be handled while scaling linearly with the size of the graph that RDFPathh can be used to investigate graph properties such as a variant of the famous six degrees of separation paradigm typically encountered in social graphs (Przyjaciel-Zablocki et al., 2011). It focuses on the path queries and studies their implementation based on MapReduce. There are various RDF query languages such as RQL, SeRQL, RDQL, Triple, N3, Versa, RxPath, RPL, and SPARQL (Przyjaciel-Zablocki et al., 2011). RDFPath has a competitive expressiveness to these other RDF query languages (Przyjaciel-Zablocki et al., 2011). A comparison of RDFPath capabilities with these other RDF query language shows that RDFPath has the same capabilities of SPARQL 1.1for the adjacent nodes, adjacent edges, the degree of a node, and fixed-length path. However, RDFPath shows more capabilities than SPARQL 1.1 in areas like the distance between two nodes and shortest paths as it has partial support for these two properties. However, SPARQL 1.1 shows full support to the aggregate functions while RDFPath shows only partial support (Przyjaciel-Zablocki et al., 2011). Table 2 shows the comparison of RDFPath with other RDF query languages including SPARQL.
Table 2: Comparison of RDF Query Language, adapted from (Przyjaciel-Zablocki et al., 2011).
2. PigSPARQL
PigSPARQL
is regarded as a competitive yet easy to
use SPARQL query processing system on MapReduce that allows ad-hoc SPARQL query processing n large RDF
graphs out of the box (Schätzle, Przyjaciel-Zablocki, Hornung, & Lausen,
2013). PigSPARQL is described as a system for
processing SPARQL queries using the MapReduce framework by translating them
into Pig Latin programs where each Pig Latin program is executed by a series of
MapReduce jobs on a Hadoop cluster (Sakr & Gaber, 2014; Schätzle et al., 2013). PigSPARQL
utilizes the query language of Pig, which is a data analysis platform on top of
Hadoop MapReduce, as an intermediate layer between SPARQL and MapReduce (Schätzle et al., 2013). That intermediate layer provides an
abstraction level which makes PigSPARQL independent of Hadoop version and
accordingly ensures the compatibility to future changes of the Hadoop framework
as they will be covered by the underlying Pig layer (Schätzle et al., 2013). This intermediate layer of Pig Latin approach
provides the sustainability of PigSPARQL and is an attractive long-term
baseline for comparing various MapReduce based SPARQL implementations which are also underpinned by the competitiveness
with the existing systems such as HadoopRDF (Schätzle et al., 2013). As illustrated in Figure 3, the PigSPARQL
workflow begins with the SPARQL that is mapped to Pig Latin by parsing the
SPARQL query to generate an abstract syntax tree which is translated into a SPARQL Algebra tree (Schätzle et al., 2013). Several optimizations are applied on the
Algebra level like the early execution of filters and a re-arrangement of
triple patterns by selectivity (Schätzle et al., 2013). The optimized Algebra tree is traversed bottom-up,
and an equivalent sequence of Pig Latin
expressions are generated for every SPARQL Algebra operator (Schätzle et al., 2013). Pig automatically maps the resulting Pig
Latin script into a sequence of MapReduce iterations at the runtime (Schätzle et al., 2013).
PigSPARQL is described as easy to use and competitive baseline for the comparison of MapReduce based SPARQL processing. PigSPARQL exceeds the functionalities of most existing research prototypes with the support of SPARQL 1.0 (Schätzle et al., 2013).
Figure 3: PigSPARQL Workflow From SPARQL to MapReduce, adapted from (Schätzle et al., 2013).
3. Interactive SPARQL Query Processing on Hadoop: Sempala
In (Schätzle, Przyjaciel-Zablocki, Neu, & Lausen, 2014), an interactive SPARQL query processing techniques “Sempala” on Hadoop is proposed. Sempala is a SPARQL-over-SQL-on-Hadoop approach designed with selective queries (Schätzle et al., 2014). It shows significant performance improvements compared to existing approaches (Schätzle et al., 2014). The approach of Sempala is inspired by the trend of applying SQL-on-Hadoop field where several new systems are developed for interactive SQL query processing such as Hive, Sharl, Presto, Phoenix, Impala and so forth (Schätzle et al., 2014). Thus, Sempala as the SPARQL-over-SQL approach is introduced to follow the trend and provide interactive-time SPARQL query processing on Hadoop (Schätzle et al., 2014). With Sempala, the data is stored in RFD in a columnar layout on HDFS and use Impala, which is an open source massive parallel processing (MPP) SQL query engine for Hadoop, to serve as the execution layer on top (Schätzle et al., 2014). The architecture of Sempala is illustrated in Figure 4.
Figure 4: Sempala Architecture
adapted from (Schätzle et al., 2014).
Two main components of the architecture of the proposed Sempala; RDF Loader and Query Compiler (Schätzle et al., 2014). The RDF Loader converts an RDF dataset into the data layout used by Sempala. The Query Compiler rewrites a given SPARQL query into the SQL dialect of Impala based on the layout of the data (Schätzle et al., 2014). The Query Compiler of Sempala is based on the algebraic representation of SPARQL expressions defined by W3C recommendation (Schätzle et al., 2014). Jena ARQ is used to parse a SPARQL query into the corresponding algebra tree (Schätzle et al., 2014). Some basic algebraic optimization such as filter pushing is applied (Schätzle et al., 2014). The final step is to traverse the tree bottom up to generate the equivalent Impala SQL expressions based on the unified property table layout (Schätzle et al., 2014). In a comparison of Sempala with other Hadoop based systems such as Hive, PigSPARQL, MapMerge, and MAPSIN. Hive is the standard SQL warehouse for Hadoop based on MapReduce (Schätzle et al., 2014). The same query with minor syntactical modification can run on the same data because Impala is developed to be highly compatible with Hive (Schätzle et al., 2014). Sempala seems to follow a similar approach as PigSPARQL. However, in PigSPARQL, the Pig is used as the underlying system and intermediate level between MapReduce and SPARQL (Schätzle et al., 2014). MapMerge is an efficient map-side merge join implementation for scalable SPARQL BGP (“BasicGraphPatterns” (W3C, 2016)) which reduces the shuffling of the data between map and reduce phases in MapReduce (Schätzle et al., 2014). MAPSIN is an approach that uses HBase, which is standard NoSQL database for Hadoop to store RDF data and applies a map-side index nested loop join which avoids the reduce phase of the MapReduce (Schätzle et al., 2014). The findings of (Schätzle et al., 2014) shows that Sempala outperforms Hive and PigSPARQL, while MapMerge and MAPSIN could not be used because they only support SPARQL BGP (Schätzle et al., 2014).
4. Map-Side Index Nested Loop Join (MAPSIN JOIN)
MapReduce is
facing the challenge of processing joins because the datasets are very large (Sakr & Gaber, 2014). Two datasets can be joined using MapReduce, but they have to be located on the same
machine, which is not practical (Sakr & Gaber, 2014). Thus, solutions such as reduce-side approach are used and regarded to be the most prominent
and flexible join technique in MapReduce (Sakr & Gaber, 2014). The reduce-side
approach is also known as “Repartition Join” because datasets at the map phase are read and repartition according to the join
key at the shuffle phase, while the actual computation for join is done in the reduce phase (Sakr & Gaber, 2014). The problem with this approach is that the
datasets are transferred through the
network with no regard to the join output which can consume a lot of the
bandwidth of the network and cause bottleneck (Sakr & Gaber, 2014; Schätzle et al., 2013). Another solution called map-side join
solution is introduced, where the actual
join processing is done in the map phase to avoid the shuffle and reduce phase
and avoid transferring both datasets over the network (Sakr & Gaber, 2014). The
most common approach is the map-side merge join, although it is hard to
cascade, in addition to the advantage of avoiding the shuffle and reduce phase
is lost (Sakr & Gaber, 2014). Thus, the MAPSIN approach is proposed which is a map-side index nested
loop join based on HBase (Sakr & Gaber, 2014; Schätzle et al., 2013). The MAPSIN join has the indexing capabilities
of HBase which improves the query performance of the selective queries (Sakr & Gaber, 2014; Schätzle et al., 2013). The capabilities retain the flexibility of
reduce-side joins while utilizing the effectiveness of a map-side join without
any modification to the underlying framework (Sakr & Gaber, 2014; Schätzle et al., 2013).
Comparing MAPSIN with PigSPARQL, MAPSIN performs faster than PigSPARQL when using a sophisticated storage schema based on HBase which works well for selective queries but diminishes significantly in performance for less selective queries (Schätzle et al., 2013). However, MAPSIN does not support the queries of LUBM (Lehigh University Benchmark (W3C, 2016). The query runtime of MAPSIN is close to the runtime of the merge join approach (Schätzle et al., 2013).
5. HadoopRDF
HadoopRDF is proposed by (Tian, Du, Wang, Ni, & Yu, 2012) to combine the advantages of high fault tolerance and high throughput of the MapReduce distributed framework and the sophisticated indexing and query answering mechanism (Tian et al., 2012). HadoopRDF is developed on Hadoop cluster with many computers and echo node in the cluster has a sesame server to supply the service for storing and retrieving the RDF data (Tian et al., 2012). HadoopRDF is a MapReduce-based RDF system which stores data directly in HDFS and does not require any modification to the Hadoop framework (Przyjaciel-Zablocki et al., 2013; Sakr & Gaber, 2014; Tian et al., 2012). The basic idea is to substitute the rudimentary HDFS without indexes and a query execution engine, with more elaborated RDF stores (Tian et al., 2012). The architecture of HadoopRDF is illustrated in Figure 5.
Figure 5: HadoopRDF Architecture,
adapted from (Tian et al., 2012).
The architecture of HadoopRDF is similar to the architecture of Hadoop which scales up to thousands of nodes (Tian et al., 2012). Hadoop framework is the core of the HadoopRDF (Tian et al., 2012). Hadoop is built on top of HDFS, which is a replicated key-value store under the control of a central NameNode (Tian et al., 2012). Files in HDFS are broken into chunks fixed size, and the replica of these chunks are distributed across a group of DataNodes (Tian et al., 2012). The NameNode tracks the size and location of each replica (Tian et al., 2012). Hadoop which is a MapReduce framework is used for the computational purpose in the data-intensive application (Tian et al., 2012). In the architecture of HadoopRDF, the RDF stores are incorporated into the MapReduce framework. HadoopRDF is an advanced SPARQL engine which splits the original RDF graph according to predicates and objects and utilizes a cost-based query execution plan for reduce-side join (Przyjaciel-Zablocki et al., 2013; Sakr & Gaber, 2014; Schätzle et al., 2013). HadoopRDF can re-balance automatically when the cluster size changes but join processing is also done in the reduce phase (Przyjaciel-Zablocki et al., 2013; Sakr & Gaber, 2014). The findings of (M. Husain et al., 2011) indicated that HadoopRDF is more scalable and handles low selectivity queries more efficiently than RDF-3X. Moreover, the result showed that HadoopRDF is much more scalable than BigOWLIM and provides more efficient queries for the large data set (M. Husain et al., 2011). HadoopRDF requires a pre-processing phase like most systems (Przyjaciel-Zablocki et al., 2013; Sakr & Gaber, 2014).
6. RDF-3X: RDF Triple eXpress
RDF-3X is proposed
by (Neumann & Weikum, 2008). The RDF-3X engine is an implementation of
SPARQL which achieves excellent performance by pursuing a RISC-style
architecture with a streamlined architecture (Neumann & Weikum, 2008). RISC
is Reduced Instruction Set Computer which is a type of microprocessor
architecture that utilizes a small, highly-optimized
set of instructions, rather than a more specialized
set of instruction often found in other types of architectures (Neumann & Weikum, 2008). Thus, RDF-3X follows the concept of
RISC-style with “reduced instruction set” designed to support RDF. RDF-3X is described to be a generic solution
for storing and indexing RDF triples that eliminates
the need for physical-design turning (Neumann & Weikum, 2008). RDF-3X provides a query optimizer for
choosing optimal join orders using a cost model based on statistical synopses
for entire join paths (Neumann & Weikum, 2008). It also provides a powerful and simple query
processor which leverage fast merge joins to the large-scale data (Neumann & Weikum, 2008). Three major components in RDF-3X; physical
design, query processor, and the query
optimizer. The physical design component
is workload-independent by creating appropriate indexes over a single “giant triples
table” (Neumann & Weikum, 2008). The query processor is RISC-style by relying
mostly on merge joins over sorted index lists.
The query optimizer focuses on join order in its generation of the execution plan (Neumann & Weikum, 2008).
The findings of (Neumann & Weikum, 2008) showed that RDF-3X addressed the challenge of schema-free data and copes very well with data that exhibit a large diversity of property names (Neumann & Weikum, 2008). The optimizer of RDF-3X is known to produce efficient query execution plan (Galarraga, Hose, & Schenkel, 2014). The RDF-3X maintains local indexes for all possible orders and combinations of the triple components and for aggregations which enable efficient local data access (Galarraga et al., 2014). RDF-3X does not support LUMB. RDF-3X is a single-node RDF-store which builds indexes over all possible permutations of subject, predicate and object (Huang, Abadi, & Ren, 2011; M. Husain et al., 2011; Schätzle et al., 2014; Zeng et al., 2013). RDF-X3 is regarded to be the fastest existing semantic web repository and state-of-the-art “benchmark” engine for single place machines (M. Husain et al., 2011; Przyjaciel-Zablocki et al., 2013). Thus, it outperforms any other solution for queries with bound objects and aggregate queries (M. Husain et al., 2011). However, the performance of RDF-3X diminishes exponentially for unbound queries and queries with even simple joins if the selectivity factor is low (M. Husain et al., 2011; Przyjaciel-Zablocki et al., 2013). The experiment of (M. Husain et al., 2011) showed that RDF-3X is not only slower for such queries, it often aborts and cannot complete the query (M. Husain et al., 2011).
7. Rya: A Scalable RDF Triple Store for the Clouds
In (Punnoose et al., 2012), the Rya is proposed as a new scalable system for storing
and retrieving RDF data in cluster nodes.
In Rya, OWL model is used as a set of triples and store them in the
triple store (Punnoose et al., 2012). Storing all the data
in the triple store provides the benefits of using Hadoop MapReduce to run
large batch processing jobs against the data set (Punnoose et al., 2012). The first phase of
the process is performed only once at the time when the OWL model is loaded
into Rya (Punnoose et al., 2012). In phase 1,
MapReduce job runs to iterate through the entire graph of relationships and
output the implicit relationships found as explicit RDF triples stores into the
RDF store (Punnoose et al., 2012). The second phase of
the process is performed every time a query is run, and once all explicit and
implicit relationships are stored in Rya, the Rya query planner can expand the
query at the runtime to utilize all these relationships (Punnoose et al., 2012). Three table index
for indexing RDF triples is used to enhance the performance (Punnoose et al., 2012). The results of (Punnoose, Crainiceanu, & Rapp, 2015) showed that Rya outperformed SHARD. Moreover, in comparison
with the graph-partitioning algorithm introduced by (Huang et al., 2011), as indicated in (Punnoose et al., 2015), the performance of Rya showed superiority in many cases
over Graph Partitioning (Punnoose et al., 2015).
Conclusion
This
project provided a survey on the state-of-the-art techniques for applying MapReduce to improve the RDF data query
processing performance. Tremendous
effort from the industry and researchers have been exerted to develop efficient
and scalable RDF processing system. The
project discussed the RDF framework and
the major building blocks of the semantic web architecture. The RDF store architecture and the MapReduce
Parallel Processing Framework and Hadoop are
discussed in this project.
Researchers have exerted effort in developing Semantic Web technologies
which have been standardized to address
the inadequacy of the current traditional
analytical techniques. This paper also
discussed the most prominent standardized
semantic web technologies RDF and
SPARQL. The project also discussed and
analyzed in details various techniques applied on MapReduce to improve the RDF
query processing performance. These techniques include RDFPath, PigSPARQL,
Interactive SPARQL Query Processing on Hadoop (Sempala), Map-Side Index Nested
Loop Join (MAPSIN JOIN), HadoopRDF, RDF-3X (RDF Triple eXpress), and Rya (a
Scalable RDF Triple Store for the Clouds).
References
Apache, J. (2017a). ARQ – A SPARQL Processor for Jena
Apache, J.
(2017b). LARQ – Adding Free Text Searches
to SPARQL
Bakshi, K.
(2012). Considerations for big data:
Architecture and approach. Paper presented at the Aerospace Conference,
2012 IEEE.
Bao, Y., Ren, L.,
Zhang, L., Zhang, X., & Luo, Y. (2012). Massive
sensor data management framework in cloud manufacturing based on Hadoop.
Paper presented at the Industrial Informatics (INDIN), 2012 10th IEEE
International Conference on.
Boussaid, O.,
Tanasescu, A., Bentayeb, F., & Darmont, J. (2007). Integration and
dimensional modeling approaches for complex data warehousing. Journal of Global Optimization, 37(4),
571. doi:10.1007/s10898-006-9064-6
Choi, H., Son,
J., Cho, Y., Sung, M. K., & Chung, Y. D. (2009). SPIDER: a system for scalable, parallel/distributed evaluation of
large-scale RDF data. Paper presented at the Proceedings of the 18th ACM
conference on Information and knowledge management.
Cloud Security
Alliance. (2013). Big Data Analytics for Security Intelligence. Big Data Working Group.
Connolly, T.,
& Begg, C. (2015). Database Systems:
A Practical Approach to Design, Implementation, and Management (6th Edition
ed.): Pearson.
De Mauro, A.,
Greco, M., & Grimaldi, M. (2015). What
is big data? A consensual definition and a review of key research topics.
Paper presented at the AIP Conference Proceedings.
Fadzil, A. F. A.,
Khalid, N. E. A., & Manaf, M. (2012). Performance
of scalable off-the-shelf hardware for data-intensive parallel processing using
MapReduce. Paper presented at the Computing and Convergence Technology
(ICCCT), 2012 7th International Conference on.
Firat, M., &
Kuzu, A. (2011). Semantic web for e-learning bottlenecks: disorientation and
cognitive overload. International Journal
of Web & Semantic Technology, 2(4), 55.
Galarraga, L.,
Hose, K., & Schenkel, R. (2014). Partout:
a distributed engine for efficient RDF processing. Paper presented at the
Proceedings of the 23rd International Conference on World Wide Web.
Hu, H., Wen, Y.,
Chua, T., & Li, X. (2014). Toward Scalable Systems for Big Data Analytics:
A Technology Tutorial. Practical
Innovation, Open Solution, 2, 652-687. doi:10.1109/ACCESS.2014.2332453
Hu, P., &
Dai, W. (2014). Enhancing fault tolerance based on Hadoop cluster. International Journal of Database Theory and
Application, 7(1), 37-48.
Huang, J., Abadi,
D. J., & Ren, K. (2011). Scalable SPARQL querying of large RDF graphs. Proceedings of the VLDB Endowment, 4(11),
1123-1134.
Husain, M.,
McGlothlin, J., Masud, M. M., Khan, L., & Thuraisingham, B. M. (2011).
Heuristics-based query processing for large RDF graphs using cloud computing. IEEE transactions on knowledge and data
engineering, 23(9), 1312-1327.
Husain, M. F.,
Khan, L., Kantarcioglu, M., & Thuraisingham, B. (2010). Data intensive query processing for large
RDF graphs using cloud computing tools. Paper presented at the Cloud
Computing (CLOUD), 2010 IEEE 3rd International Conference on.
Inukollu, V. N.,
Arsi, S., & Ravuri, S. R. (2014). Security Issues Associated with Big Data
in Cloud Computing. International Journal
of Network Security & Its Applications, 6(3), 45.
doi:10.5121/ijnsa.2014.6304
Jinquan, D., Jie,
H., Shengsheng, H., Yan, L., & Yuanhao, S. (2012). The Hadoop Stack: New
Paradigm for Big Data Storage and Processing. Intel Technology Journal, 16(4), 92-110.
Khan, N., Yaqoob,
I., Hashem, I. A. T., Inayat, Z., Ali, M. W. K., Alam, M., . . . Gani, A.
(2014). Big Data: Survey, Technologies, Opportunities, and Challenges. The Scientific World Journal, 1-18.
doi:10.1155/2014/712826
Konstantinou, N.,
Spanos, D.-E., Stavrou, P., & Mitrou, N. (2010). Technically approaching
the semantic web bottleneck. International
Journal of Web Engineering and Technology, 6(1), 83-111.
Krishnan, K.
(2013). Data warehousing in the age of
big data: Newnes.
Lee, K.-H., Lee,
Y.-J., Choi, H., Chung, Y. D., & Moon, B. (2012). Parallel data processing
with MapReduce: a survey. ACM SIGMOD
Record, 40(4), 11-20.
Mishra, B. S. P.,
Dehuri, S., & Kim, E. (2016). Techniques
and Environments for Big Data Analysis: Parallel, Cloud, and Grid Computing
(Vol. 17): Springer.
Modoni, G. E.,
Sacco, M., & Terkaj, W. (2014). A
survey of RDF store solutions. Paper presented at the Engineering,
Technology and Innovation (ICE), 2014 International ICE Conference on.
Myung, J., Yeon,
J., & Lee, S.-g. (2010). SPARQL basic
graph pattern processing with iterative MapReduce. Paper presented at the Proceedings
of the 2010 Workshop on Massive Data Analytics on the Cloud.
Neumann, T.,
& Weikum, G. (2008). RDF-3X: a RISC-style engine for RDF. Proceedings of the VLDB Endowment, 1(1),
647-659.
Nicolaidis, I.,
& Iniewski, K. (2017). Building
Sensor Networks: CRC Press.
Polato, I., Ré,
R., Goldman, A., & Kon, F. (2014). A comprehensive view of Hadoop
research—A systematic literature review. Journal
of Network and Computer Applications, 46, 1-25.
Przyjaciel-Zablocki,
M., Schätzle, A., Hornung, T., & Lausen, G. (2011). Rdfpath: Path query processing on large rdf graphs with mapreduce.
Paper presented at the Extended Semantic Web Conference.
Przyjaciel-Zablocki,
M., Schätzle, A., Skaley, E., Hornung, T., & Lausen, G. (2013). Map-side merge joins for scalable SPARQL BGP
processing. Paper presented at the Cloud Computing Technology and Science
(CloudCom), 2013 IEEE 5th International Conference on.
Punnoose, R.,
Crainiceanu, A., & Rapp, D. (2012). Rya:
a scalable RDF triple store for the clouds. Paper presented at the
Proceedings of the 1st International Workshop on Cloud Intelligence.
Punnoose, R.,
Crainiceanu, A., & Rapp, D. (2015). SPARQL in the cloud using Rya. Information Systems, 48, 181-195.
Sakr, S., &
Gaber, M. (2014). Large Scale and big
data: Processing and Management: CRC Press.
Schätzle, A.,
Przyjaciel-Zablocki, M., Hornung, T., & Lausen, G. (2013). PigSPARQL: a SPARQL query processing
baseline for big data. Paper presented at the Proceedings of the 12th
International Semantic Web Conference (Posters & Demonstrations
Track)-Volume 1035.
Schätzle, A.,
Przyjaciel-Zablocki, M., Neu, A., & Lausen, G. (2014). Sempala: interactive SPARQL query processing on hadoop. Paper
presented at the International Semantic Web Conference.
Sun, Y., &
Jara, A. J. (2014). An extensible and active semantic model of information
organizing for the Internet of Things. Personal
and Ubiquitous Computing, 18(8), 1821-1833. doi:10.1007/s00779-014-0786-z
Tian, Y., Du, J.,
Wang, H., Ni, Y., & Yu, Y. (2012). Hadooprdf: A scalable rdf data analysis
system. 8th ICIC, 633-641.
Tiwana, A., &
Balasubramaniam, R. (2001). Integrating knowledge on the web. IEEE Internet Computing, 5(3), 32-39.
White, T. (2012-important-buildingblocks).
Hadoop: The definitive guide: ”
O’Reilly Media, Inc.”.
Yang, H.-c.,
Dasdan, A., Hsiao, R.-L., & Parker, D. S. (2007). Map-reduce-merge: simplified relational data processing on large
clusters. Paper presented at the Proceedings of the 2007 ACM SIGMOD
international conference on Management of data.
Zeng,
K., Yang, J., Wang, H., Shao, B., & Wang, Z. (2013). A distributed graph engine for web scale RDF data. Paper presented
at the Proceedings of the VLDB Endowment.
Hadoop was developed by Yahoo and Apache to run jobs
in hundreds of terabytes of data (Yan, Yang, Yu, Li, & Li, 2012). A
various large corporation such as Facebook, Amazon have used Hadoop as
it offers high efficiency, high scalability, and high reliability (Yan et al., 2012). Hadoop has faced various
limitation such as low-level programming paradigm and schema, strictly batch
processing, time skew and incremental computation (Alam & Ahmed, 2014). The incremental computation is
regarded to be one of the major shortcomings of Hadoop technology (Alam & Ahmed, 2014). The efficiency on handling
incremented data is at the expense of losing the incompatibility with
programming models which are offered by
non-incremental systems such as MapReduce, which requires the implementation of
incremental algorithms and increasing the
complexity of the algorithm and the code (Alam & Ahmed, 2014). The caching technique is
proposed by (Alam & Ahmed, 2014) as a solution. This caching
solution will be at three levels; the Job, the Task and the Hardware (Alam & Ahmed, 2014).
Incoop is another solution proposed by (Bhatotia, Wieder, Rodrigues, Acar, &
Pasquin, 2011). The Incoop proposed solution is to extend
the open-source implementation of Hadoop of MapReduce programming paradigm to
run unmodified MapReduce program in an incremental method (Bhatotia et al., 2011; Sakr & Gaber,
2014). Incoop allows programmers to increment the
MapReduce programs automatically without any modification to the code (Bhatotia et al., 2011; Sakr & Gaber,
2014). Moreover, information about the previously
executed MapReduce tasks are recorded by Incoop to be reused in subsequent
MapReduce computation when possible (Bhatotia et al., 2011; Sakr & Gaber,
2014).
The Incoop is not a perfect solution, and it has some shortcomings which are
addressed by (Sakr & Gaber, 2014; Zhang, Chen,
Wang, & Yu, 2015). Some enhancements are implemented to Incoop
to include incremental HDFS called Inc-HDFS, Contraction Phase, and “Memoization-aware
Scheduler” (Sakr & Gaber, 2014). The Inc-HDFS provides the delta
technique in the inputs of two consecutive job runs and splits the input based
on the contents where the compatibility with HDFS is maintained. The
Contraction phase is a new phase in the MapReduce framework consisting of
breaking up the Reduce tasks into smaller sub-computation
forming an inverted tree allowing the small portion of the input changes
to the path from the corresponding leaf
to the root to be computed (Sakr & Gaber, 2014). The Memoization-aware Scheduler
is a modified version of the scheduler of
Hadoop taking advantage of the locality of memorized results (Sakr & Gaber, 2014).
Another solution called i2MapReduce proposed by (Zhang et al., 2015) which was compared to Incoop by (Zhang et al., 2015). The i2MapReduce does
not perform the task-level computation but rather
a key-value pair level incremental processing. This solution also supports more complex
iterative computation, which is used in data mining and reduces the I/O
overhead by applying various techniques (Zhang et al., 2015). IncMR is an enhanced framework
for the large-scale incremental data processing (Yan et al., 2012). It inherits the simplicity of
the standard MapReduce, it does not modify HDFS
and utilizes the same APIs of the MapReduce (Yan et al., 2012). When using IncMR, all programs
can complete incremental data processing without any modification (Yan et al., 2012).
In conclusion, various efforts are exerted by
researchers to overcome the incremental computation limitation of Hadoop, such
as Incoop, Inc-HDFS, i2MapReduce, and IncMR. Each proposed solution is an attempt to
enhance and extend the standard Hadoop to avoid overheads such as I/O, to
increase the efficiency, and without increasing the complexing of the
computation and without causing any modification to the code.
References
Alam, A., & Ahmed, J. (2014). Hadoop architecture and its issues.
Paper presented at the Computational Science and Computational Intelligence
(CSCI), 2014 International Conference on.
Bhatotia, P.,
Wieder, A., Rodrigues, R., Acar, U. A., & Pasquin, R. (2011). Incoop: MapReduce for incremental
computations. Paper presented at the Proceedings of the 2nd ACM Symposium
on Cloud Computing.
Sakr, S., &
Gaber, M. (2014). Large Scale and big
data: Processing and Management: CRC Press.
Yan, C., Yang,
X., Yu, Z., Li, M., & Li, X. (2012). Income:
Incremental data processing based on MapReduce. Paper presented at the
Cloud Computing (CLOUD), 2012 IEEE 5th International Conference on.
Zhang,
Y., Chen, S., Wang, Q., & Yu, G. (2015). i^2MapReduce: Incremental
MapReduce for Mining Evolving Big Data. IEEE
transactions on knowledge and data engineering, 27(7), 1906-1919.
The standard MapReduce framework faced the challenge
of the iterative computation which is required
in various operations such as data mining, PageRank, network traffic analysis, graph
analysis, social network analysis, and so forth (Bu, Howe, Balazinska, & Ernst, 2010;
Sakr & Gaber, 2014). These analyses techniques require the data to
be processed iteratively until the computation satisfies a convergence or
stropping condition (Bu et al., 2010; Sakr & Gaber, 2014). Due to this limitation, and to
this critical requirement, this iterative process is implemented and executed
manually using a driver program when using the standard MapReduce framework (Bu et al., 2010; Sakr & Gaber, 2014). However, the manual
implementation and execution of such iterative computation have two major
problems (Bu et al., 2010; Sakr & Gaber, 2014). The first problem is reflected
in loading unchanged data from iteration to iteration wasting input/output
(I/O), network bandwidth, and CPU resources (Bu et al., 2010; Sakr & Gaber, 2014). The second problem is reflected in the overhead of the termination
condition when the output of the application did
not change for two consecutive iterations and reached a fixed point (Bu et al., 2010; Sakr & Gaber, 2014). This termination condition may
require an extra MapReduce job on each iteration which causes overhead for
scheduling extra tasks, reading extra data from disk, and moving data across
the network (Bu et al., 2010; Sakr & Gaber, 2014).
Researchers exerted efforts to
solve the iterative computation.
HaLoop is proposed by (Bu et al., 2010), and Twister by (Ekanayake et al., 2010), Pregel by (Malewicz et al., 2010). One solution to the iterative
computation limitation, as the case in HaLoop by (Bu et al., 2010) and Twister by (Ekanayake et al., 2010) are to identify and keep invariant
data during the iterations, where reading unnecessary data repeatedly is avoided. The HaLoop by (Bu et al., 2010) implemented two caching functionalities (Bu et al., 2010; Sakr & Gaber, 2014). The first caching technique is implemented on
the invariant data in the first iteration and reusing them in a later iteration. The second caching technique
is implemented on the outputs of reducer making the check for the fixpoint more
efficient without adding any extra MapReduce job (Bu et al., 2010; Sakr & Gaber, 2014).
The solution of Pregel by (Malewicz et al., 2010) is more focused on the graph and was inspired by the Bulk Synchronous
Parallel model (Malewicz et al., 2010). This solution provides the
synchronous computation and communication (Malewicz et al., 2010) and uses explicit messaging approach to acquire remote information and
does not replicate remote values locally (Malewicz et al., 2010). Mahoot is another solution that
was introduced to solve the iterative computing by grouping a series of chained
jobs to obtain the results (Polato, Ré, Goldman, & Kon, 2014). In Mahoot solution, the result
of each job is pushed into the next job until the final results are obtained (Polato et al., 2014). The iHadoop proposed by (Elnikety, Elsayed, & Ramadan, 2011) schedules iterations asynchronously and connects the output of one
iteration to the next allowing both to process their data concurrently (Elnikety et al., 2011). The task scheduler of the iHadoop utilizes the inter-iteration data
locality by scheduling tasks that exhibit a producer/consumer relation on the
same physical machine allowing a fast transfer of the local data (Elnikety et al., 2011).
Apache Hadoop and Apache Spark are the most popular
technology for the iterative computation using in-memory data processing engine
(Liang, Li, Wang, & Hu, 2011). Hadoop defines the iterative
computation as a series of MapReduce jobs where each job reads the data from Hadoop Distributed File System (HDFS)
independently, processes the data, and writes the data back to HDFS (Liang et al., 2011). Dacoop was proposed by Liang as an extension
to Hadoop to handle the data-iterative applications, by using cache technique
for repeatedly data processing and introducing shared memory-based data cache
mechanism (Liang et al., 2011). The iMapReduce is another solution proposed by (Zhang, Gao, Gao, & Wang, 2012) to provide support of iterative processing implementing the persistent
tasks of the map and reduce during the
whole iterative process and how the persistent tasks are terminated (Zhang et al., 2012). The iMapReduce avoid three major overheads. The first overhead is the job startup
overhead which is avoided by building an
internal loop from reduce to map within a
job. The second overhead is the communication overhead which is avoided by separating the iterated state
data from the static structure data. The
third overhead is the synchronization overhead which is avoided by allowing
asynchronous map task execution (Zhang et al., 2012).
References
Bu, Y., Howe, B., Balazinska, M.,
& Ernst, M. D. (2010). HaLoop: Efficient iterative data processing on large
clusters. Proceedings of the VLDB
Endowment, 3(1-2), 285-296.
Ekanayake, J.,
Li, H., Zhang, B., Gunarathne, T., Bae, S.-H., Qiu, J., & Fox, G. (2010). Twister: a runtime for iterative MapReduce.
Paper presented at the Proceedings of the 19th ACM international symposium on
high performance distributed computing.
Elnikety, E.,
Elsayed, T., & Ramadan, H. E. (2011). iHadoop:
asynchronous iterations for MapReduce. Paper presented at the Cloud
Computing Technology and Science (CloudCom), 2011 IEEE Third International
Conference on.
Liang, Y., Li,
G., Wang, L., & Hu, Y. (2011). Dacoop:
Accelerating data-iterative applications on Map/Reduce cluster. Paper
presented at the Parallel and Distributed Computing, Applications, and
Technologies (PDCAT), 2011 12th International Conference on.
Malewicz, G.,
Austern, M. H., Bik, A. J., Dehnert, J. C., Horn, I., Leiser, N., &
Czajkowski, G. (2010). Pregel: a system
for large-scale graph processing. Paper presented at the Proceedings of the
2010 ACM SIGMOD International Conference on Management of data.
Polato, I., Ré,
R., Goldman, A., & Kon, F. (2014). A comprehensive view of Hadoop
research—A systematic literature review. Journal
of Network and Computer Applications, 46, 1-25.
Sakr, S., &
Gaber, M. (2014). Large Scale and big
data: Processing and Management: CRC Press.
Zhang,
Y., Gao, Q., Gao, L., & Wang, C. (2012). imapreduce: A distributed
computing framework for iterative computation. Journal of Grid Computing, 10(1), 47-68.
In 2004, Google introduced MapReduce framework as a
Parallel Processing framework which deals with large set of data (Bakshi, 2012; Fadzil, Khalid, &
Manaf, 2012; White, 2012). The MapReduce framework has gained much
popularity because it has features for hiding sophisticated operations of the
parallel processing (Fadzil et al., 2012). Various MapReduce frameworks
such as Hadoop were introduced because of the enthusiasm towards MapReduce (Fadzil et al., 2012).
The capability of the MapReduce framework was
realized by different research areas such as data warehousing, data mining, and
the bioinformatics (Fadzil et al., 2012). MapReduce framework consists of
two main layers; the Distributed File System (DFS) layer to store data and the
MapReduce layer for data processing (Lee, Lee, Choi, Chung, & Moon, 2012;
Mishra, Dehuri, & Kim, 2016; Sakr & Gaber, 2014). DFS is a major feature of the MapReduce framework
(Fadzil et al., 2012).
MapReduce framework is using large clusters of
low-cost commodity hardware to lower the cost (Bakshi, 2012; H. Hu, Wen, Chua, & Li,
2014; Inukollu, Arsi, & Ravuri, 2014; Khan et al., 2014; Krishnan, 2013;
Mishra et al., 2016; Sakr & Gaber, 2014; White, 2012). MapReduce framework is using “Redundant
Arrays of Independent (and inexpensive) Nodes (RAIN),” whose components are
loosely coupled and when any node goes down there is no negative impact on the
MapReduce job (Sakr & Gaber, 2014; Yang, Dasdan,
Hsiao, & Parker, 2007). MapReduce framework involves the
“Fault-Tolerance” by applying the replication technique and allows replacing
any crashed nodes with another node without affecting the current running job (P. Hu & Dai, 2014-7; Sakr &
Gaber, 2014). MapReduce framework involves the automatic
support for the parallelization of execution which makes the MapReduce highly
parallel and yet abstracted (P. Hu & Dai, 2014-7; Sakr &
Gaber, 2014).
The Top Three Features of
Hadoop
The Hadoop
Distributed File System (HDFS) is one of the major components of the Hadoop
framework for storing large files (Bao, Ren, Zhang, Zhang, & Luo, 2012;
CSA, 2013; De Mauro, Greco, & Grimaldi, 2015) and allowing access to data scattered over
multiple nodes in without any exposure to the complexity of the environment (Bao et al., 2012; De Mauro et al., 2015). The MapReduce programming model is another major component
of the Hadoop framework (Bao et al., 2012; CSA, 2013;
De Mauro et al., 2015)
which is designed to implement the distributed and parallel algorithms
efficiently (De Mauro et al., 2015). HBase is the third
component of Hadoop framework (Bao et al., 2012). HBase is developed
on the HDFS and is a NoSQL (Not only SQL) type database (Bao et al., 2012).
The key features of Hadoop include
the scalability and flexibility, cost efficiency and fault tolerance (H. Hu et al., 2014; Khan et
al., 2014; Mishra et al., 2016; Polato, Ré, Goldman, & Kon, 2014; Sakr
& Gaber, 2014). Hadoop allows the nodes in the cluster to
scale up and down based on the computation requirements and with no change in
the data formats (H. Hu et al., 2014; Polato
et al., 2014). Hadoop also provides massively parallel
computation to commodity hardware decreasing the cost per terabyte of storage
which makes the massively parallel computation affordable when the volume of
the data gets increased (H. Hu et al., 2014). The Hadoop technology
offers the flexibility feature as it is not tight with a schema which allows
the utilization of any data either structured, non-structures, and semi-structured, and the aggregation of the
data from multiple sources (H. Hu et al., 2014; Polato
et al., 2014). Hadoop also allows nodes to crash without
affecting the data processing. It
provides fault tolerance environment where data and computation can be
recovered without any negative impact on the processing of the data (H. Hu et al., 2014; Polato
et al., 2014; White, 2012).
Pros and Cons of MapReduce Framework
MapReduce was introduced to solve the problem of parallel processing of a
large set of data in a distributed
environment which required manual management of the hardware resources (Fadzil et al., 2012; Sakr & Gaber, 2014).
The complexity of the parallelization is
solved by using two techniques:
Map/Reduce technique, and Distributed File System (DFS) technique (Fadzil et al., 2012; Sakr & Gaber, 2014).
The parallel framework must be reliable to ensure good resource
management in the distributed environment using off-the-shelf hardware to solve
the scalability issue to support any future requirement for processing (Fadzil et al., 2012).
The earlier frameworks such as the Message Passing Interface (MPI)
framework was having a reliability issue and had a fault-tolerance issue when processing a large set of data (Fadzil et al., 2012).
MapReduce framework covers the two categories of the scalability; the
structural scalability, and the load scalability (Fadzil et al., 2012).
It addresses the structural scalability by using the DFS which allows
forming large virtual storage for the framework by adding off-the-shelf
hardware. MapReduce framework addresses
the load scalability by increasing the number of the nodes to improve the
performance (Fadzil et al., 2012).
However, the earlier version of
MapReduce framework faced challenges. Among these challenges are the join operation and the lack of support
for aggregate functions to join multiple datasets
in one task (Sakr & Gaber, 2014). Another limitation
of the standard MapReduce framework is found
in the iterative processing which is required
for analysis techniques such as PageRank algorithm, recursive relational
queries, and social network analysis (Sakr & Gaber, 2014). The standard
MapReduce does not share the execution of work to reduce the overall amount of
work (Sakr & Gaber, 2014). Another limitation
was found in the lack of support of data index and column storage but support
only for a sequential method when
scanning the input data. Such a lack of data index affected the query
performance (Sakr & Gaber, 2014). Moreover, many
argued that MapReduce is not regarded to be the optimal solution for structured
data. It is known as shared-nothing
architecture, which supports scalability (Bakshi, 2012; Jinquan, Jie,
Shengsheng, Yan, & Yuanhao, 2012; Sakr & Gaber, 2014; White, 2012),
and the processing of large unstructured data sets (Bakshi, 2012). MapReduce has the
limitation of performance and efficiency (Lee et al., 2012).
References
Bakshi, K. (2012, 3-10 March 2012). Considerations for big data: Architecture
and approach. Paper presented at the Aerospace Conference, 2012 IEEE.
Bao, Y., Ren, L.,
Zhang, L., Zhang, X., & Luo, Y. (2012). Massive
sensor data management framework in cloud manufacturing based on Hadoop.
Paper presented at the Industrial Informatics (INDIN), 2012 10th IEEE
International Conference on.
CSA, C. S. A.
(2013). Big Data Analytics for Security Intelligence. Big Data Working Group.
De Mauro, A.,
Greco, M., & Grimaldi, M. (2015). What
is big data? A consensual definition and a review of key research topics.
Paper presented at the AIP Conference Proceedings.
Fadzil, A. F. A.,
Khalid, N. E. A., & Manaf, M. (2012). Performance
of scalable off-the-shelf hardware for data-intensive parallel processing using
MapReduce. Paper presented at the Computing and Convergence Technology
(ICCCT), 2012 7th International Conference on.
Hu, H., Wen, Y.,
Chua, T.-S., & Li, X. (2014). Toward scalable systems for big data
analytics: A technology tutorial. IEEE
Access, 2, 652-687.
Hu, P., &
Dai, W. (2014-7). Enhancing fault tolerance based on Hadoop cluster. International Journal of Database Theory and
Application, 7(1), 37-48.
Inukollu, V. N.,
Arsi, S., & Ravuri, S. R. (2014). Security issues associated with big data
in cloud computing. International Journal
of Network Security & Its Applications, 6(3), 45.
Jinquan, D., Jie,
H., Shengsheng, H., Yan, L., & Yuanhao, S. (2012). The Hadoop Stack: New
Paradigm for Big Data Storage and Processing. Intel Technology Journal, 16(4), 92-110.
Khan, N., Yaqoob,
I., Hashem, I. A. T., Inayat, Z., Mahmoud Ali, W. K., Alam, M., . . . Gani, A.
(2014). Big Data: Survey, Technologies, Opportunities, and Challenges. The Scientific World Journal, 2014.
Krishnan, K.
(2013). Data warehousing in the age of
big data: Newnes.
Lee, K.-H., Lee,
Y.-J., Choi, H., Chung, Y. D., & Moon, B. (2012). Parallel data processing
with MapReduce: a survey. ACM SIGMOD
Record, 40(4), 11-20.
Mishra, B. S. P.,
Dehuri, S., & Kim, E. (2016). Techniques
and Environments for Big Data Analysis: Parallel, Cloud, and Grid Computing
(Vol. 17): Springer.
Polato, I., Ré,
R., Goldman, A., & Kon, F. (2014). A comprehensive view of Hadoop
research—A systematic literature review. Journal
of Network and Computer Applications, 46, 1-25.
Sakr, S., &
Gaber, M. (2014). Large Scale and big
data: Processing and Management: CRC Press.
White, T. (2012).
Hadoop: The definitive guide: ”
O’Reilly Media, Inc.”.
Yang,
H.-c., Dasdan, A., Hsiao, R.-L., & Parker, D. S. (2007). Map-reduce-merge: simplified relational data
processing on large clusters. Paper presented at the Proceedings of the
2007 ACM SIGMOD international conference on Management of data.
Keywords: World Wide Web, Web, Performance
Bottlenecks, Large-Scale Data.
Introduction
The
journey of the Web starting with Web 1.0 is known as the “Web of Information
Connections,” followed by Web 2.0 which is is known as “Web of People
Connections” (Aghaei,
Nematbakhsh, & Farsani, 2012). Web 3.0 is known as “Web of Knowledge
Connections, “Web 4.0 is known as the “Web of Intelligence Connections” (Aghaei
et al., 2012).
Web 5.0 is known as “Symbionet Web” (Patel,
2013).
This project discusses
this journey of the Web starting from Web 1.0 until Web 5.0. The inception of the World Wide Web (W3)
known as the “Web” goes back to 1989 when Berners-Lee introduced it through a
project for CERN. The underlying concept
behind the Web is hypertext paradigm which links documents together. The project was the starting point to change
the way we communicate with each other.
Web
1.0 is the first generation of the Web, and it was read-only, with no
interaction with the users. It was used to broadcast information to the
users. It used the simple framework of
Client/Server which is known as a single point of failure. Web 2.0 was introduced in 2004 by Tim
O’Reilly as a platform which allows read-write and interaction of users. The topology of Web 2.0 is Peer-To-Peer to
avoid the single point of the failure in Web 1.0. All nodes are serving as server and client
and have the same capabilities to respond to users requests. The topology of Web 2.0 Peer-to-Peer is
called Master/Slave. Web 3.0 was introduced
by Markoff of New York Times in 2006 and is known as the Semantic Web. Berners-Lee introduced the concept of the
Semantic Web in 2001. The Semantic Web
has a layered architecture including URIs, RDF, Ontology Web Language (OWL),
XML and other components. Web 4.0 and
Web 5.0 are still in progress. Web 4.0
is known as the “Intelligent Web,” while Web 5.0 is known as “Symbionet
Web.” It is expected that the Artificial
Intelligence will play a key role in Web 4.0, and consequently Web 5.0. The project addressed the main sources for
the large-scale data in each Web generation.
Moroever, the key technologies and the underlying architecture and
framework for each Web generation are also discussed in this paper.
The project also discussed and
analyzed the bottleneck and the performance of the Web for each Web generation
from Client/Server simple topology of Web 1.0 to Peer-To-Peer of Web 2.0
topology, to the layered topology of Semantic Web of Web 3.0. Moreover, the bottleneck is also discussed
for Web 4.0 using the Internet of Things technology. Each generation has added tremendous value to
our lives and how we communicate with each.
Web 1.0 – The Universal
Access to Read-Only Static Pages
Web
1.0 is the first generation of the World Wide Web (W3), known as the Web (Aghaei
et al., 2012; Choudhury, 2014; Kujur & Chhetri, 2015; Patel, 2013).
The inception of the W3 project took place in CERTN in 1989 to enhance
the effectiveness of the CERN communication system (Kujur
& Chhetri, 2015).
Berners-Lee realized this hypertext paradigm could be applied globally (Kujur
& Chhetri, 2015). The W3 project allowed access to the
information online (T.
J. Berners-Lee, 1992).
The term “Web” for the world goes to the similarities for the
construction of a spider (T.
J. Berners-Lee, 1992).
The
underlying concept of W3 was using hypertext paradigm, through which documents
are referring to each using links (T.
J. Berners-Lee, 1992).
The user can access the document from that link universally (T.
J. Berners-Lee, 1992).
The user can create documents and link them into the Web using the
hypertext (T.
J. Berners-Lee, 1992).
If the data is stored in the database, the server can be modified to
access the database from W3 clients and present it to the Web, as the case with
the generic Oracle server using the SQL SELECT statement (T.
J. Berners-Lee, 1992).
Large sets of structured data such as the database cannot be handled by
Web 1.0 hypertext alone (T.
J. Berners-Lee, 1992).
The solution for this limitation is to add “search” functionality to the
hypertext paradigm (T.
J. Berners-Lee, 1992).
The indexes, which are regarded to be “special documents,” can be used
for the search where the user can provide a keyword that results in that
“special document” or “index” which has a link to the documents found as a
result of that keyword (T.
J. Berners-Lee, 1992). The
phone book was the first document which was published on the Web (T.
Berners-Lee, 1996; T. J. Berners-Lee, 1992).
Berners-Lee
is regarded to be the innovator of the Web 1.0 (Aghaei
et al., 2012; Choudhury, 2014; Kujur & Chhetri, 2015). Web 1.0 is defined as “It is an information
space in which the items of interest referred to as resources are identified by
a global identifier called as Unifrom Resource Identifiers (URIs)” (Kujur
& Chhetri, 2015; Patel, 2013). Web
1.0 was read-only passive Web with no interaction with the websites (Choudhury,
2014).
The content of the data and the data
management are the sole responsibility of the webmaster. In the
1990s, the data sources included digital technology and database systems which
organizations widely adopted storing a large amount of data, such as bank
trading transactions, shopping mall records, and government sector archives (Hu,
Wen, Chua, & Li, 2014). Companies such as Google and Yahoo began to
develop search functions and portals to information for users (Kambil,
2008). Web 1.0 lasted from 1989 until 2005 (Kujur
& Chhetri, 2015; Patel, 2013). Example of Web 1.0 includes the “Britannica
Online” which provide information for read-only (Loretz,
2017).
Key
Technologies of Web 1.0
The key protocols
for Web 1.0 are HyperText Markup Language (HTML), HyperText Transfer Protocol
(HTTP), and Universal Resource Identifier (URI) (Choudhury,
2014; Patel, 2013).
New protocols include XML,
XHTML. The Cascading Style Sheet (CSS)
Server-Side Scripting include ASP, PHP, JSP, CGI, and Perl (Patel,
2013). The Client-Side Scripting include JavaScript,
VBScript, and Flash (Patel,
2013).
Web
1.0 Architecture
The underlying
architecture of the Web 1.0 is Client/Server topology where the Server
retrieves data responding to the client requests where the browser is residing (T.
J. Berners-Lee, 1992). A common library of the information access
code of the network is shared by the clients
(T.
J. Berners-Lee, 1992). The servers existed at the time of W3 project
were Files, VMS/Help, Oracle, and GNU Info (T.
J. Berners-Lee, 1992).
The Client/Server topology is described as a single point of failure as
there is a total dependency on the server (Markatos,
2002).
Web
1.0 Performance Bottleneck
Web
1.0 framework consists of a web server, clients which are connected to the
server using the internet (Mosberger
& Jin, 1998).
HTTP is the protocol that connects the client and the server (Mosberger
& Jin, 1998).
In (Mosberger
& Jin, 1998), httperf tool was used to test the
load and the performance of the web server which responds to several client requests
(Mosberger
& Jin, 1998).
The httperf tool has two main goals (Mosberger
& Jin, 1998). The first goal is about the good
and predictable performance. The second
goal is about the “ease of extensibility” (Mosberger
& Jin, 1998).
Load sustainability is a key factor in a good performance of the web
server (Mosberger
& Jin, 1998).
There are various client performance limits which should not be regarded
as server performance limits (Mosberger
& Jin, 1998).
The client CPU imposes a limitation on the client (Mosberger
& Jin, 1998).
The size of the Transmission Transfer Protocol (TCP) port space whose
numbers are sixteen bits wide. Privilege
process reserve 1,024 of the 64K available ports. The port cannot be reused until it gets
expired. Thus, the TCP TIME_WAIT plays a role as its expired state allows the
port to be reused (Mosberger
& Jin, 1998). This scenario can cause a serious limit and
bottleneck for client sustainable offered rate (Mosberger
& Jin, 1998).
With one minute timeout, the sustainable rate is about 1,075 requests
per seconds. However, with the recommended value of RFC-793 to be four-minute time
out, the maximum rate would be 268 requests per second instead (Mosberger
& Jin, 1998).
The total and the per-process number of file descriptors that can be
opened are limited in most operating systems (Mosberger
& Jin, 1998).
The “per-process limits” is ranged from 256 to 2,048. The file descriptor cannot be used until it is
closed, the httperf timeout value plays a role in the number of the open file
descriptors. If the client confronts
with a bottleneck, the operating system of the client can be tuned to increase
the limit of the open file descriptors (Mosberger
& Jin, 1998).
The TCP connection typically has
a “socket receive” and “send buffer” (Mosberger
& Jin, 1998).
The clients loads are limited to the clients’ memory available for the
“socket receive” (Mosberger
& Jin, 1998).
Concurrent TCP connections can cause a bottleneck and poor performance (Mosberger
& Jin, 1998).
For
the server side performance, the granularity of the process scheduling in
operating systems is measured in millisecond ranges and plays a key role in the
performance of the server responding to several requests from several clients (Mosberger
& Jin, 1998).
Tools such as httperf check the network activity for input and output
using select() functions and monitor the real-time using gettimeofday()
functions (Mosberger
& Jin, 1998).
The limited number of the ephemeral ports is ranged from 1,024 to 5,000
which can cause a problem when running out of ports (Mosberger
& Jin, 1998).
Tools such as httperf re-use the ports as soon as they are
released. However, the incompatibility
of the TCP between Unix and NT broke this solution where Unix allows pre-empting
the TIME_WAIT state, while NT did not allow it during the arrival of the SYN
segment (Mosberger
& Jin, 1998).
Allocating the ports using round-robin method solves this problem. Several thousand TCP control blocks can
cause slow system calls (Mosberger
& Jin, 1998). The hash table is usually used to look up the
TCP control blocks for incoming network traffic is standard for execution time (Mosberger
& Jin, 1998).
However, some BSD-derived systems still use linear control block search
for the bind() and connect() system calls, which can increase the slow system
calls. The solution was found when
system closes the connection. Thus,
tools such as httperf applied the concept of closing the connection by using
RESET (Mosberger
& Jin, 1998).
Web 2.0 – The Universal
Access to Read-Write Web Pages
In 2004, Dale Dougherty the Vice-President of O’Reilly Media defined Web 2.0 as read-write web (Aghaei et al., 2012; Choudhury, 2014; Kujur & Chhetri, 2015; Patel, 2013). Web 2.0 is defined by Tim O’Reilly as cited in (Aghaei et al., 2012; Choudhury, 2014; Kujur & Chhetri, 2015; Miller, 2008) as follows : “Web 2.0 is the business revolution in the computer industry caused by the move to the internet as platform, and an attempt to understand the rules for success on that new platform. Chief among those rules is this: Build applications that harness network effects to get better the more people use them.” Others defined Web 2.0 as a “Transition” from Web 1.0 where information is isolated to computing platforms that are interlinked together which function as a local software for the user (Miller, 2008). In an attempt to differentiate between Cloud Computing and Web 2.0, Tim stated as cited in (Miller, 2008) “Cloud computing refers specifically to the use of the Internet as a computing platform; Web 2.0, as I’ve defined it, is an attempt to explore and explain the business rules of that platform.”
Web 2.0 has shifted Web 1.0 not only from read-only to be read-write but also to be technology centric, business-centric and user-centric (Choudhury, 2014). The technology-centric is found in the platform concept of the Web 2.0 which is different from a client/server framework of Web 1.0. The platform technology is associated with blogs, wikis, and Really Simple Syndication (RSS) feeds (Choudhury, 2014). The business-centric concept is reflected in the shift to the internet as a platform and comprehending the key success factors using this new platform concept on the internet (Choudhury, 2014). The user-centric concept is the shift from companies publishing content for read, to communities of users who are interacting and communicating with each other using the new platform on the internet (Choudhury, 2014). Tim O’Reilly identified a list of the differences between Web 1.0 and Web 2.0 (O’Reilly, 2007) (see Figure 1).
Figure 1: Web 1.0 vs. Web 2.0 Examples (O’Reilly, 2007).
Web
2.0 has other attributes such as “wisdom web,” “people-centric web,” and
“participative web” with reading the writing capabilities (Aghaei
et al., 2012; Choudhury, 2014; Kujur & Chhetri, 2015; Patel, 2013).T With Web 2.0, the user can have
flexible web design, updates, collaborative content creation and
modification (Aghaei
et al., 2012; Patel, 2013). The support for collaboration is one of the
major characteristics of Web 2.0, where people can share data (Patel,
2013).
Examples of Web 2.0 implementation
include a social network such as MySpace, Facebook, Twitter, media sharing such
as Youtube (Patel,
2013). Thus, the data for Web 2.0 is generated from
all these resources of MySpace, Facebook, Twitter and so force. With
Web 2.0, the data is growing very fast and entering a new level of “Petabyte
age” (Demirkan
& Delen, 2013).
Key
Technologies of Web 2.0
Web
2.0 utilized key technologies to allow people to communicate with each other
through the new platform on the internet (Aghaei
et al., 2012).
The key technologies of Web 2.0 include The RSS, Blogs, Mashups, Tags,
Folksonomy, and Tag Clouds (Aghaei
et al., 2012).
Three development approaches to create Web 2.0 applications: Asynchronous JavaScript and XML (AJAX), Flex
and Google Web Toolkit (Aghaei
et al., 2012).
Web
2.0 Architecture
Web 2.0 has
various architecture patterns (Governor,
Hinchcliffe, & Nickull, 2009). Three main levels of the Web 2.0 architecture
patterns starting from the most concrete to the most abstract which is
high-level design pattern (Governor
et al., 2009).
Some of these Web 2.0 architecture patterns include Service-Oriented
Architecture (SOA), Software as a Service (SaaS), Participation-Collaboration,
Asynchronous Particle Update, Mashup,
Rich User Experience (RUE), the Synchronized Web, Collaborative Tagging, Declarative
Living and Tag Gardening, Semantic Web Grounding, Persistent Rights Management,
and Structured Information (Governor
et al., 2009).
Web
2.0 Performance Bottleneck
In Web 1.0, the
Client/Server architecture provided users to access the data through the
internet. However, the users had the
experience of “wait” (Miller,
2008). As discussed earlier, the client has
bottlenecks in addition to the server’s bottleneck. The issue in Web 1.0 is that all
communications among computers had to go through the server first (Miller,
2008). Due to the requirement of having every client
passes through the server first, the concept of Peer-to-Peer was established to
solve this overload and bottleneck on the server side. Web 2.0 works with Peer-to-Peer framework (Aghaei
et al., 2012; Patel, 2013). While in Web 1.0, the server has the full
responsibility and capabilities to respond to clients, in Web 2.0 using the
Peer-to-Peer computing, each computer has the same responsibilities and
capabilities as the server (Miller,
2008). This new relationship between computers is
referred to as master/slave where the central sever acts as the master and the
client computers act as the slave (Miller,
2008). In Peer-to-Peer framework in Web 2.0, each
computer servers as a client as well as a server (Miller,
2008).
Peer-To-Peer framework provided the
capability of streaming live video from a single source to a large number of
receivers or peers over the internet without any special support from the
network (Magharei
& Rejaie, 2006, 2009). This capability is called P2P streaming
mechanism (Magharei
& Rejaie, 2006, 2009). There are two main bottlenecks with a P2P
streaming mechanism (Magharei
& Rejaie, 2006, 2009). The first bottleneck is called the “bandwidth
bottleneck,” and the second bottleneck is called “content bottleneck” (Magharei
& Rejaie, 2006, 2009). The “bandwidth bottleneck” is experienced by
the peer when the aggregate bandwidth available from all other peers is not
sufficient to fully utilize the incoming access link bandwidth (Magharei
& Rejaie, 2006, 2009). The “content bottleneck” is experienced by
the peer when the useful content from other peers is not sufficient to fully
utilize the bandwidth available in the network (Magharei
& Rejaie, 2006, 2009).
The discussion on the bottleneck in
Web 2.0 is not limited to Peer-to-Peer but also to platform computing. In Web
2.0, the user interacts with the web as it is not only read as the case in Web
1.0 but also has the write capabilities.
This feature of read-write capabilities can cause a bottleneck with a
high concurrent reading and writing operations using a large set of data (Choudhury,
2014). A data-intensive application or very
large-scale data transfer can cause a bottleneck, and it can be very costly (Armbrust
et al., 2009).
To solve this issue is to send disks or even whole computers via
overnight delivery services (Armbrust
et al., 2009).
When the data is moved to the Cloud, the data does not have any
bottleneck any longer because the data transfer is within the Cloud such as
storing the data in S3 in Amazon Web
Services, which can be transferred without any bottleneck to EC2 (Elastic
Compute Cloud) (Armbrust
et al., 2009).
WAN bandwidth can cause a bottleneck, and the intra-cloud networking
technology can also have a performance bottleneck (Armbrust
et al., 2009).
One Gigabit Ethernet (1GbE) reflects a bandwidth that can have a
bottleneck because it is not sufficient to process a large set of data using
technology such as Map/Reduce. However,
in the Cloud Computing, the 10Gigabit Ethernet is used for such aggregation
links (Armbrust
et al., 2009).
Map/Reduce is a processing
strategy to divide the operation into two jobs Map and then Reduce using the
“filtering-join-aggregation” tasks (Ji,
Li, Qiu, Awada, & Li, 2012). When using the classic Hadoop MapReduce, the
cluster is not artificially segregated into Map and reduce slots (Krishnan,
2013). Thus, the application jobs are bottlenecked
on the Reduce operation which limits the scalability in the job execution (Krishnan,
2013).
Because of the scalability bottleneck faced by the traditional Map/Reduce,
Yahoo introduced YARN for Yet Another Resource Negotiators to overcome such
scalability bottleneck in 2010 (White,
2012).
There are additional performance
issues which cannot be predicted. For
instance, there is a performance degradation when using Virtual Machines (VMs) which
share CPU and main memory in the Cloud Computing platform (Armbrust
et al., 2009). Moreover, the bottleneck at the computing
platform can be caused when moving a large set of data continuously to a remote
CPUs (Foster,
Zhao, Raicu, & Lu, 2008). The Input/Output (I/O) operations can also
cause a performance degradation issue (Armbrust
et al., 2009; Ji et al., 2012). Flash
memory is another aspect that can minimize the performance. The scheduling of VMs for High-Performance
Computing (HPC) apps is another unpredictable performance issue (Armbrust
et al., 2009).
The issue of such scheduling for HPC is to ensure that all threads of a
program are running concurrently which is not provided by either VMs or the
operating systems (Armbrust
et al., 2009).
Another issue with the computing platform about bottleneck is the
availability of the cloud environment is threatened when there is a flooding attack
which will affect the available bandwidth, processing power and the memory (Fernandes,
Soares, Gomes, Freire, & Inácio, 2014). Thus, to minimize the bottleneck issue when
processing a large set of data in the computing platform, the data must be
distributed over many computers (Foster
et al., 2008; Modi, Patel, Borisaniya, Patel, & Rajarajan, 2013).
Web 3.0 – Semantic Web
Web 3.0 is the
third generation of the Web. It was
introduced by John Markoff of the New York Times in 2006 (Aghaei
et al., 2012).
Web 3.0 is known as “Semantic Web” as well as the “Web of Cooperation” (Aghaei
et al., 2012), and as “Executable Web (Choudhury,
2014; Kujur & Chhetri, 2015; Patel, 2013). Berners-Lee introduced the concept of Semantic
Web in 2001 (T.
Berners-Lee, Hendler, & Lassila, 2001). The underlying concept of Web 3.0 is to link,
integrate and analyze data from various datasets to obtain new information
stream (Aghaei
et al., 2012).
Web 3.0 has a variety of capabilities (Aghaei
et al., 2012).
When using Web 3.0, the data management can be improved, the accessibility
of the mobile internet can be supported, creativity and innovation are
simulated, the satisfaction of the customers is enhanced, and the collaboration
in the social web is organized (Aghaei
et al., 2012; Choudhury, 2014). Another key factor for Web 3.0 is that the
web is no longer understood only by human but also by machines (Aghaei
et al., 2012; Choudhury, 2014). In other words, the web is understood by human
and machines in Web 3.0, where the machines first understand the web followed
by a human (Aghaei
et al., 2012; Choudhury, 2014). Web 3.0 supports world wide database and web oriented
architecture which was described as a web of document (Patel,
2013).
Web 3.0 characteristics include portable
personal web, and consolidating dynamic content, lifestream, individuals, RDF,
and user engagements (Aghaei
et al., 2012; Patel, 2013). Examples of Web 3.0 include Google Map, My
Yahoo (Patel,
2013). Since Web 3.0 is described as the mobile and
sensor-based era, the majority of the data sources are from mobile and
sensor-based devices (Chen,
Chiang, & Storey, 2012).
Key
Technologies of Web 3.0
While
in Web 2.0 the content creativity of users is the target, in Web 3.0 the linked
data sets are the target. Web 3.0 is not only for publishing data on the web
but also linking related data (Choudhury,
2014). Linked Data principles introduced by
Berners-Lee as the rules to publish and connect data on the web in 2007 (Aghaei
et al., 2012; Choudhury, 2014). These rules are summarized below:
URI should be used as Names of Things. HTTP URIs should be used to look up those
Names.
Useful Information should be provided
using the standards of RDF, SPARQL by looking up the URIs.
Links to other URIs should be included to
discover more things.
In (T.
Berners-Lee et al., 2001) the technology of
the Semantic Web included XHTML, SVG, and SMIL and placed on top of the XML
layer. XSLT is used for the
transformation engines, while XPath and XPointer are used for path and pointer
engines (T.
Berners-Lee et al., 2001). CSS and XSL are used for the style engines
and formatters (T.
Berners-Lee et al., 2001).
Web
3.0 Architecture
The
Semantic Web framework is a multi-layered architecture. The degree of the structure among objects is
based on a model called Resource Description Framework” (RDF) (Aghaei
et al., 2012; Patel, 2013). The structure of the semantic data include
from the bottom up, the Unicode and URI at the bottom of the framework,
followed by the Extensible Markup Language (XML), RDF, RDF Schema, Ontology Web
Language (OWL), Logic and Proof and Trust at the top (see Figure 2).
Figure 2: Web 3.0 – Semantic Web Layered Architecture (Patel, 2013).
There
are eight categories for Web Semantic identified by (T.
Berners-Lee et al., 2001) to describe the
relation of Web Semantic with Hypermedia Research. The first category describes the basic node,
link and anchor data mode. The second
category reflects the typed nodes, links, and anchors. The Conceptual Hypertext is the third
category of this relation. Virtual Links
and Anchors is the fourth category, while searching and querying is the fifth
category. Versioning and Authentication
features, Annotation and User Interface Design beyond the navigational
hypermedia reflect the last three categories of the relation of Semantic Web
with Hypermedia Research (T.
Berners-Lee et al., 2001).
Web
3.0 is expected to include four major drivers in accordance with Steve Wheeler
as cited in (Chisega-Negrila,
2016). The first driver includes the distributed
computing. The second driver includes
the extended smart phone technology. The third driver includes the
collaborative intelligence. The last
driver is the 3D visualization and interaction (Chisega-Negrila,
2016).
Web
3.0 Performance Bottleneck
The result of the research in (Firat & Kuzu, 2011) found that the components of the Semantic Web of XML, RDF and OWL help overcome hypermedia bottlenecks in various areas of the eLearning such as the cognitive overload, disorientation in hypermedia (Firat & Kuzu, 2011). However, Web 3.0 faces the bottleneck of the search in sensor networks (Nicolaidis & Iniewski, 2017), as the use of mobile technology has been increasing (see Figure 3) (Nicolaidis & Iniewski, 2017).
Figure 3: Increasing Use of the Mobile Technology (Nicolaidis & Iniewski, 2017)
The
wireless communication in sensor networks is causing the bottleneck with the
increasing number of requests (Nicolaidis
& Iniewski, 2017).
Thus, the time to answer queries gets increased, which result in
decreasing the search experience of the users (Nicolaidis
& Iniewski, 2017).
Thus, the approach of the push-based is used to force sensors to push
regularly their new readings to a base station, which decrease the latency to
the users’ requests (Nicolaidis
& Iniewski, 2017). However, this approach cannot
guarantee that the data is up-to-date as it can be outdated (Nicolaidis
& Iniewski, 2017).
If there are no much changes, the update can be sent when there is a
change. However, if the changes are too often, this approach can cause
congestion of the wireless channel resulting in delayed or missing messages (Nicolaidis
& Iniewski, 2017).
Prediction-Model-Based approaches are proposed to reduce the volume of
the data when transmitting dynamic sensor data (Nicolaidis
& Iniewski, 2017).
However, to create an accurately predicted model, a series of sensor
readings need to be transmitted (Nicolaidis
& Iniewski, 2017).
Using the prediction approach, the latency is reduced because the
prediction at a based station instead of contacting a sensor over the non-reliable
multi-hop connection (Nicolaidis
& Iniewski, 2017).
Moreover, scaling the systems to more sensors causes another bottleneck
which needs to be solved by utilizing the distribution approach (Nicolaidis
& Iniewski, 2017).
Moreover,
in (Konstantinou,
Spanos, Stavrou, & Mitrou, 2010), the contemporary
Semantic Web is defined as the counterpart of the Knowledge Acquisition
bottleneck where it was too expensive to acquire and encode the large amount of
knowledge that is needed for the application (Konstantinou
et al., 2010).
The annotation of the content in Web Semantic is still an open issue and
is regarded as an obstacle for Semantic Web applications which need the considerable
volume of data to demonstrate their utility (Konstantinou
et al., 2010).
Web 4.0 – Intelligent Web
The rapid increase in the communication using wireless enables another major transition in the Web (Kambil, 2008). This transition enables people to connect with objects anywhere and anytime at the physical world as well as at the virtual world (Kambil, 2008). Web 4.0 is the fourth generation of Web and is known as “web of intelligence connections” (Aghaei et al., 2012) or “Ultra-Intelligent Electronic Agent” (Choudhury, 2014; Kujur & Chhetri, 2015; Patel, 2013). It is read-write, execution, and concurrency web with intelligent interactions (Aghaei et al., 2012; Patel, 2013). Some consider it as “Symbiotic Web” where human and machines can interact in symbiosis fashion (Aghaei et al., 2012; Choudhury, 2014; Kujur & Chhetri, 2015; Patel, 2013) and as “Ubiquitous Web” (Kujur & Chhetri, 2015; Patel, 2013; Weber & Rech, 2009). Using Web 4.0, machines will be intelligent reading the web content and deliver web pages with superior performance at the real-time (Aghaei et al., 2012; Choudhury, 2014). It is also known as WebOS which will act as the “Middleware” which will act as an operating system (Aghaei et al., 2012; Choudhury, 2014). WebOS is expected to resemble or be equivalent to our brain which will highly interact intelligently (Aghaei et al., 2012; Choudhury, 2014). As indicated in (Aghaei et al., 2012) “the web is moving toward using artificial intelligence to become an intelligent web.” Web 4.0 will reflect the integration between people and virtual worlds and objects at the real-time (Kambil, 2008). The Artificial Intelligence technologies are expected to play a role in Web 4.0 (Weber & Rech, 2009).
The major challenges for Web 4.0
generating value that is based on full integration of the physical objects and
virtual objects with other contents that are generated by users (Kambil,
2008). This challenge could lead to the next
generation of Supervisory Control and Data Acquisition (SCADA) applications (Kambil,
2008). The challenge can also lead to generating
value from sources such as “entertainment” which collect information from human
and objects (Kambil,
2008).
The migration of virtual world to
physical is another challenge in Web 4.0 (Patel,
2013). A good example of Web 4.0 is provided by (Patel,
2013),
which is to search or Google your home to locate an object such as your car
key.
An application of the Web 4.0 is
implemented by Rafi Haladjian and Olivier who created the first consumer
electronics in Amazon, which can recognize you and provide recommended product
and personalized advice (Patel,
2013). The time frame for Web 4.0 is expected to be
2020 – 2030 (Loretz,
2017; Weber & Rech, 2009). Web
4.0 is still in progress.
Web 4.0 and “Internet of Things”
Some like (Loretz, 2017) considers “Internet of Things” as part of Web 3.0 and Web 4.0, while others like (Pulipaka, 2016) categorize it under Web 4.0. Thus, the discussion on the “Internet of Things” from the bottleneck perspective is addressed under the Web 4.0 section of this paper.
As indicated in (Atzori, Iera, & Morabito, 2010) Internet of Things (IoT) is regarded to be “one of the most promising
fuels of Big Data expansion” (De Mauro, Greco, & Grimaldi, 2015). IoT seems promising as Google
acquired Nest for $3.2 billion in January 2014 (Dalton, 2016). The
Nest is a smart hub producer at the forefront of the Internet of Things (Dalton, 2016). This acquiring can tell the
importance of the IoT. IoT is becoming
powerful because it affects our daily life and the behavior of the users (Atzori et al., 2010). The underlying concept of the
IoT is the ubiquitous characteristics that using various devices such as
sensors, mobile phones and so forth (Atzori et al., 2010).
As cited in (Batalla & Krawiec, 2014) “Internet of Things (IoT) is global
network infrastructure, linking physical and virtual objects through the exploitation
of data capture and communication capabilities” (Batalla & Krawiec, 2014). IoT is described by (Batalla & Krawiec, 2014) as “a huge connectivity platform for self-managed objects.” IoT is increasingly growing, and the reasons for such strong growth go to
the inexpensive cost of the computing including sensors and the growth of Wi-Fi
(Gholap & Asole, 2016; Gubbi, Buyya,
Marusic, & Palaniswami, 2013), and 4G-LTE (Gubbi et al., 2013). Other factors include the growth of mobiles, the rise of software
developments, the emergence of standardized low-power wireless technologies (Gholap & Asole, 2016).
With the advancement in the Web, from static web pages in Web 1.0 to network web in Web 2.0, to ubiquitous
computing web in Web 3.0, the increased requirement for “data-on-demand” using complex, and intuitive queries becomes
significant (Gubbi et al., 2013). With IoT, many objects and many
things surrounding people will be on the network (Gubbi et al., 2013). The Radio Frequency IDentification (RFID) and the technologies of
the sensor network emerge to respond to the IoT network challenges where
information and communication systems are embedded in the environment around us
invisibly (Gubbi et al., 2013). The computing criterion for the
IoT will go beyond the traditional scenarios of the mobile computing which
utilize the smartphones and portables (Gubbi et al., 2013). IoT will evolve to connect existing everyday objects and embed
intelligence into our environment (Gubbi et al., 2013).
Web 4.0 and IoT Performance Bottleneck
The elements of IoT include the RFID, Wireless Sensor Networks (WSN),
Addressing Schemes, Data Storage and Analytics, and Visualization (Gubbi et al., 2013). IoT will require the persistence
of the network to channel the traffic of the data ubiquitously. IoT confronts a bottleneck at the interface
between the gateway and wireless sensor devices (Gubbi et al., 2013). The bottleneck at the interface is between the Internet and smart
object networks of the RFID or WSN subnets (Jin, Gubbi, Marusic, & Palaniswami,
2014). Moreover, the scalability of the address of
the device of the existing network must be sustainable (Gubbi et al., 2013). The performance of the network or
the device functioning should not be affected by adding networks and devices (Gubbi et al., 2013; Jin et al., 2014). The Uniform Resource Name (URN) system will play a significant role in the development of IoT to overcome these issues (Gubbi et al., 2013).
Moreover, although Cloud can enhance and simplify the communication of
IoT, the Cloud can still represent a bottleneck in certain scenarios (Botta, de Donato, Persico, & Pescapé,
2016). As indicated in (Gubbi et al., 2013), the high capacity and large-scale web data generated by IoT and as IoT
grows, the Cloud becomes a bottleneck (Gubbi et al., 2013). A framework proposed by (Gubbi et al., 2013) to enable scalability of the cloud to provide the capacity that is required for IoT. While the proposed
framework of (Gubbi et al., 2013) enables the separation of the networking, computation, storage and
visualization theme, it allows the independent growth in each domain, at the
same time enhances each other in an environment that is shared among them (Gubbi et al., 2013).
Web 4.0 and IoT New Challenges
IoT faces additional challenges such as Addressing and Networking Issues (Atzori et al., 2010). The investigation effort has been exerted about the integration of RFID tags
into IPv6. Mobile IP is proposed as a
solution for the mobility in IoT scenarios (Atzori et al., 2010). Moreover, the DNS (domain name servers), which provide IP address of a
host from a certain input name, does not seem to serve the IoT scenarios where
communications are among objects and not hosts. Object Name Service (ONS) is
proposed as a solution to the DNS issue (Atzori et al., 2010). ONS will associate a reference
to a description of the object and the related RFID tag identifier, and it must
work in a bidirectional manner (Atzori et al., 2010). For the complex operation of
IoT, the Object Code Mapping Service (OCMS) is still an open issue (Atzori et al., 2010). TCP as the Transmission Control
Protocol is found inadequate and
inefficient for the transmission control of end-to-end in the IoT (Atzori et al., 2010). The TCP issue is still an open
issue for IoT (Atzori et al., 2010). Other issues of IoT include
Quality of Service, Security, and
Privacy.
Web 5.0 – Symbionet Web
Web 5.0 is still in progress and can be regarded as “Symbionet Web” (Loretz, 2017; Patel, 2013) or “Telepathic Web” (Loretz, 2017). In Web 5.0, people will be able to have their own Personal Servers (PS), where they can store and communicate with their personal data using Smart Communicator (SC) such as Smart Phones, Tablets and so forth (Patel, 2013). The Smart Communications will be 3D Virtual World of the Symbionet (Patel, 2013). Web 5.0 will be aware of your emotions and feelings (Kambil, 2008). Objects such as tools such as headset are investigated for emotional interaction (Kambil, 2008). While there is a claim from some companies that they map emotions and feelings, this claim can be hard to imagine because emotions and feelings are complex (Kambil, 2008). However, some technologies are examining the emotions effect (Kambil, 2008). There is an idea that there will “brain implant” which enables the person to communicate with the internet and web by thoughts (Loretz, 2017). The person will be able to open pages just by thoughts (Loretz, 2017). The time frame for Web 5.0 is after 2030 (Loretz, 2017).
Conclusion
This
project discussed the Web from the inception of Web 1.0 until the last generation
of Web 5.0. The project addressed the main
characteristics of each generation and the main sources for generating the
large-scale web data. Web 1.0 is known
as the “Web of Information Connections” where information is broadcasted by
companies for users to read. Web 2.0 is
known as the “Web of People Connections” where people connected. Web 3.0 is known as the “Web of Knowledge
Connections” where people share knowledge.
Web 4.0 is known as the “Web of Intelligence Connections” where
Artificial Intelligence is expected to play a role. Web 5.0 is known as the “Symbionet Web” where
emotions and feelings are expected to be communicated to the machines, and be
part of the Web interactions.
The
project also discussed the key technologies and the underlying architecture of
each Web generation. Moreover, this paper also discussed and analyzed the
performance bottlenecks when accessing the large-scale data for each Web
generation, and the proposed solutions for some of these bottlenecks and the open
issues.
References
Aghaei, S., Nematbakhsh,
M. A., & Farsani, H. K. (2012). Evolution of the world wide web: From WEB
1.0 TO WEB 4.0. International Journal of
Web & Semantic Technology, 3(1), 1.
Armbrust,
M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., . . .
Stoica, I. (2009). Above The Clouds: A Berkeley View of Cloud Computing.
Atzori,
L., Iera, A., & Morabito, G. (2010). The internet of things: A survey. Computer networks, 54(15), 2787-2805.
Batalla,
J. M., & Krawiec, P. (2014). Conception of ID layer performance at the
network level for Internet of Things. Personal
and Ubiquitous Computing, 18(2), 465-480.
Berners-Lee,
T. (1996). WWW: Past, present, and future. Computer,
29(10), 69-77.
Berners-Lee,
T., Hendler, J., & Lassila, O. (2001). The semantic web.
Berners-Lee,
T. J. (1992). The world-wide web. Computer
networks and ISDN systems, 25(4-5), 454-459.
Botta,
A., de Donato, W., Persico, V., & Pescapé, A. (2016). Integration of Cloud
Computing and Internet Of Things: a Survey. Future
Generation computer systems, 56, 684-700.
Chen,
H., Chiang, R. H., & Storey, V. C. (2012). Business Intelligence and
Analytics: From Big Data to Big Impact. MIS
quarterly, 36(4), 1165-1188.
Chisega-Negrila,
A. M. (2016, 2016). IMPACT OF WEB 3.0 ON
THE EVOLUTION OF LEARNING, Bucharest.
Choudhury,
N. (2014). World Wide Web and its journey from web 1.0 to web 4.0.
Dalton,
C. (2016). Brilliant Strategy for
Business: How to plan, implement and evaluate strategy at any level of
management: Pearson UK.
De
Mauro, A., Greco, M., & Grimaldi, M. (2015). What is big data? A consensual definition and a review of key research
topics. Paper presented at the AIP Conference Proceedings.
Demirkan,
H., & Delen, D. (2013). Leveraging the capabilities of service-oriented
decision support systems: Putting analytics and big data in cloud. Decision Support Systems, 55(1),
412-421.
Fernandes,
D. A., Soares, L. F., Gomes, J. V., Freire, M. M., & Inácio, P. R. (2014).
Security issues in cloud environments: a survey. International Journal of Information Security, 13(2), 113-170.
doi:/10.1007/s10207-013-0208-7
Firat,
M., & Kuzu, A. (2011). Semantic web for e-learning bottlenecks:
disorientation and cognitive overload. International
Journal of Web & Semantic Technology, 2(4), 55.
Foster,
I., Zhao, Y., Raicu, I., & Lu, S. (2008). Cloud Computing and Grid Computing 360-Degree Compared. Paper
presented at the 2008 Grid Computing Environments Workshop.
Gholap,
K. K., & Asole, S. (2016). Today’s Impact of Big Data on Cloud. International Journal of Engineering
Science, 3748.
Governor,
J., Hinchcliffe, D., & Nickull, D. (2009). Web 2.0 Architectures: What entrepreneurs and information architects
need to know: ” O’Reilly Media, Inc.”.
Gubbi,
J., Buyya, R., Marusic, S., & Palaniswami, M. (2013). Internet of Things
(IoT): A vision, architectural elements, and future directions. Future Generation computer systems, 29(7),
1645-1660.
Hu,
H., Wen, Y., Chua, T.-S., & Li, X. (2014). Toward scalable systems for big
data analytics: A technology tutorial. IEEE
Access, 2, 652-687.
Ji,
C., Li, Y., Qiu, W., Awada, U., & Li, K. (2012). Big Data Processing in Cloud Computing Environments. Paper
presented at the 2012 12th International Symposium on Pervasive Systems,
Algorithms and Networks.
Jin,
J., Gubbi, J., Marusic, S., & Palaniswami, M. (2014). An information
framework for creating a smart city through internet of things. IEEE Internet of Things Journal, 1(2),
112-121.
Kambil,
A. (2008). What is your Web 5.0 strategy? Journal
of business strategy, 29(6), 56-58.
Konstantinou,
N., Spanos, D.-E., Stavrou, P., & Mitrou, N. (2010). Technically
approaching the semantic web bottleneck. International
Journal of Web Engineering and Technology, 6(1), 83-111.
Krishnan,
K. (2013). Data warehousing in the age of
big data: Newnes.
Kujur,
P., & Chhetri, B. (2015). Evolution of World Wide Web: Journey From Web 1.0
to Web 4.0 IJCST, 6(1).
Magharei,
N., & Rejaie, R. (2006). Understanding
mesh-based peer-to-peer streaming. Paper presented at the Proceedings of
the 2006 international workshop on Network and operating systems support for
digital audio and video.
Magharei,
N., & Rejaie, R. (2009). Prime: Peer-to-peer receiver-driven mesh-based
streaming. IEEE/ACM Transactions on
Networking (TON), 17(4), 1052-1065.
Markatos,
E. P. (2002). Tracing a large-scale peer
to peer system: an hour in the life of gnutella. Paper presented at the
Cluster Computing and the Grid, 2002. 2nd IEEE/ACM International Symposium on.
Miller,
M. (2008). Cloud Computing, Web-Based Applications That Change the Way You Work
and Collaborate Online. Michael Miller.
Modi,
C., Patel, D., Borisaniya, B., Patel, A., & Rajarajan, M. (2013). A survey
on security issues and solutions at different layers of Cloud computing. The Journal of Supercomputing, 63(2),
561-592.
Mosberger,
D., & Jin, T. (1998). httperf—a tool for measuring web server performance. ACM SIGMETRICS Performance Evaluation
Review, 26(3), 31-37.
Nicolaidis,
I., & Iniewski, K. (2017). Building
Sensor Networks: CRC Press.
O’Reilly,
T. (2007). What is web 2.0.
Patel,
K. (2013). Incremental journey for World Wide Web: introduced with Web 1.0 to
recent Web 5.0–a survey paper.
Weber,
S., & Rech, J. (2009). An overview and differentiation of the evolutionary
steps of the web XY movement: the web before and beyond 2.0. Handbook of Research on Web 2.0, 3.0, and X.
0: Technologies, Business, and Social Applications: Technologies, Business, and
Social Applications, 12.
White, T. (2012). Hadoop:
The definitive guide: ” O’Reilly Media, Inc.”.
Internet of Things (IoT)
is regarded to be “one of the most promising fuels of Big Data expansion” (De
Mauro, Greco, & Grimaldi, 2015). IoT seems promising as Google acquired Nest
for $3.2 billion in January 2014 (Dalton, 2016). The
Nest is a smart hub producer at the forefront of the Internet of Things (Dalton, 2016). This acquiring can tell the
importance of the IoT. IoT is becoming
powerful because it affects our daily life and the behavior of the users (Atzori, Iera, & Morabito, 2010). The underlying concept of the
IoT is the ubiquitous characteristics that using various devices such as
sensors, mobile phones and so forth (Atzori et al., 2010).
As cited in (Batalla & Krawiec, 2014) “Internet of Things (IoT) is a global network infrastructure, linking
physical and virtual objects through the exploitation of data capture and
communication capabilities” (Batalla & Krawiec, 2014). IoT is described by (Batalla & Krawiec, 2014) as “a huge connectivity platform for self-managed objects.” IoT is increasingly growing, and the reasons for such strong growth go to
the inexpensive cost of the computing including sensors and the growth of Wi-Fi
(Gholap & Asole, 2016; Gubbi, Buyya,
Marusic, & Palaniswami, 2013), and 4G-LTE (Gubbi et al., 2013). Other factors include the growth of mobiles, the rise of software
developments, the emergence of standardized low-power wireless technologies (Gholap & Asole, 2016).
With the advancement in the Web, from static web
pages in Web 1.0 to network web in Web
2.0, to ubiquitous computing web in Web 3.0, the increased requirement for “data-on-demand”
using complex, and intuitive queries
becomes significant (Gubbi et al., 2013). With IoT, many objects and many
things surrounding people will be on the network (Gubbi et al., 2013). The Radio Frequency IDentification (RFID) and the technologies of
the sensor network emerge to respond to the IoT network challenges where
information and communication systems are embedded in the environment around us
invisibly (Gubbi et al., 2013). The computing criterion for the
IoT will go beyond the traditional scenarios of the mobile computing which
utilize the smartphones and portables (Gubbi et al., 2013). IoT will evolve to connect existing everyday objects and embed intelligence
into our environment (Gubbi et al., 2013).
Performance Bottleneck
The elements of IoT include the RFID, Wireless Sensor Networks (WSN), Addressing Schemes, Data Storage and Analytics, and Visualization (Gubbi et al., 2013). IoT will require the persistence of the network to channel the traffic of the data ubiquitously. IoT confronts a bottleneck at the interface between the gateway and wireless sensor devices (Gubbi et al., 2013). The bottleneck at the interface is between the Internet and smart object networks of the RFID or WSN subnets (Jin, Gubbi, Marusic, & Palaniswami, 2014). Moreover, the scalability of the address of the device of the existing network must be sustainable (Gubbi et al., 2013). The performance of the network or the device functioning should not be affected by adding networks and devices (Gubbi et al., 2013; Jin et al., 2014). The Uniform Resource Name (URN) system will play a significant role in the development of IoT to overcome these issues (Gubbi et al., 2013).
Moreover, although Cloud can enhance and simplify
the communication of IoT, the Cloud can still represent a bottleneck in certain
scenarios (Botta, de Donato, Persico, & Pescapé,
2016). As indicated in (Gubbi et al., 2013), the high capacity and large-scale web data generated by IoT and as IoT grows,
the Cloud becomes a bottleneck (Gubbi et al., 2013). A framework proposed by (Gubbi et al., 2013) to enable scalability of the cloud to provide the capacity that is required for IoT. While the proposed
framework of (Gubbi et al., 2013) enables the separation of the networking, computation, storage and
visualization theme, it allows the independent growth in each domain, at the
same time enhance each other in an environment that is shared among them (Gubbi et al., 2013).
IoT New Challenges
IoT faces additional challenges such as Addressing and Networking Issues (Atzori et al., 2010). The investigation effort has been exerted about the integration of RFID tags into IPv6. Mobile IP is proposed as a solution for the mobility in IoT scenarios (Atzori et al., 2010). Moreover, the DNS (domain name servers), which provide IP address of a host from a certain input name, does not seem to serve the IoT scenarios where communications are among objects and not hosts. Object Name Service (ONS) is proposed as a solution to the DNS issue (Atzori et al., 2010). ONS will associate a reference to a description of the object and the related RFID tag identifier, and it must work in a bidirectional manner (Atzori et al., 2010). For the complex operation of IoT, the Object Code Mapping Service (OCMS) is still an open issue (Atzori et al., 2010). TCP as the Transmission Control Protocol is found inadequate and inefficient for the transmission control of end-to-end in the IoT (Atzori et al., 2010). The TCP issue is still an open issue for IoT (Atzori et al., 2010). Other issues of IoT include Quality of Service, Security, and Privacy.
In conclusion, IoT is a promising domain that will change
how we communicate and how we live our everyday life.
There are still major issues such as the bottleneck when dealing with
large-scale of data, addressing and network issues such as DNS, and TCP. Various efforts are exerted in the research
field and industry to address these issues.
However, there are still open issues that are under investigation, which
place the IoT in the “work in progress” status, and it is not fully matured.
References
Atzori, L., Iera, A., &
Morabito, G. (2010). The internet of things: A survey. Computer networks, 54(15), 2787-2805.
Batalla, J. M.,
& Krawiec, P. (2014). Conception of ID layer performance at the network
level for Internet of Things. Personal
and Ubiquitous Computing, 18(2), 465-480.
Botta, A., de
Donato, W., Persico, V., & Pescapé, A. (2016). Integration of Cloud
Computing and Internet Of Things: a Survey. Future
Generation computer systems, 56, 684-700.
Dalton, C.
(2016). Brilliant Strategy for Business:
How to plan, implement and evaluate strategy at any level of management:
Pearson UK.
De Mauro, A.,
Greco, M., & Grimaldi, M. (2015). What
is big data? A consensual definition and a review of key research topics.
Paper presented at the AIP Conference Proceedings.
Gholap, K. K.,
& Asole, S. (2016). Today’s Impact of Big Data on Cloud. International Journal of Engineering
Science, 3748.
Gubbi, J., Buyya,
R., Marusic, S., & Palaniswami, M. (2013). Internet of Things (IoT): A
vision, architectural elements, and future directions. Future Generation computer systems, 29(7), 1645-1660.
Jin,
J., Gubbi, J., Marusic, S., & Palaniswami, M. (2014). An information
framework for creating a smart city through internet of things. IEEE Internet of Things Journal, 1(2),
112-121.
As indicated in (Berners-Lee, 1996), the Web was intended as “universe of accessible network information… Universal access means that you put it on the Web and you can access it from anywhere.” There is one space for being a universe, which is represented in the URL (Uniform Resource Locators) space which starts with HTTP. The web allowed the hypertext links to point to anything (Berners-Lee, 1996). The URLs are the most common type of Uniform Resource Identifiers (URIs) (Berners-Lee, Hendler, & Lassila, 2001). The resources are identified by global identifiers called Universal Document Identifier (UDI) (Berners-Lee, 1996) (now called Uniform Resource Identifiers) (URI) (Jacobs & Walsh, 2004). The benefits of the URIs include linking, bookmarking, caching, and indexing by search engines (Jacobs & Walsh, 2004). URIs have different schemes such as HTTP, FTP, LDAP to access the identified resource (Jacobs & Walsh, 2004). The communication between agents over the network of resources involves URIs, messages, and data (Jacobs & Walsh, 2004). This communication can use any of the Web protocol of The HyperText Transfer Protocol (HTTP), FTP (File Transfer Protocol), SOAP (Simple Object Access Protocol), and SMPT (Simple Mail Protocol Transfer) (Jacobs & Walsh, 2004). The HTTP URI scheme is defined regarding TCP-based HTTP servers (Jacobs & Walsh, 2004).
Web 1.0 is the first generation of
the Web (Aghaei, Nematbakhsh, &
Farsani, 2012; Choudhury, 2014; Patel, 2013). In Web 1.0, the
access was read-only (Aghaei et al., 2012;
Choudhury, 2014; Patel, 2013). Web 1.0 is regarded as a web of cognition (Aghaei et al., 2012). It started with a place of information for business to
publish the information publicly (Aghaei et al., 2012). In Web 1.0, no user
interaction was available, and the users were allowed to search the information
only and read it. The information was
published using static HTML (HyperText Markup
Language) (Aghaei et al., 2012;
Choudhury, 2014; Patel, 2013). The core protocols
for Web 1.0 included HTTP, HTML, and URI (Aghaei et al., 2012;
Choudhury, 2014; Patel, 2013). Web 1.0 had three
major limitations. The first limitation is reflected in the pages that can only be understood by humans (web readers) where
there is no machine compatible content (Choudhury, 2014; Patel,
2013). The second limitation reflected the content
of the websites that can only be updated
and managed solely by the webmasters who use framesets (Choudhury, 2014; Patel,
2013). The third limitation reflected the lack of
dynamic representation to acquire only static information where no web console
to perform dynamic events (Choudhury, 2014).
The second generation of Web is Web
2.0, which was introduced in 2004 (Aghaei et al., 2012). It is distinguished
from Web 1.0 as it is read-write web and interactive unlike Web 1.0, and it is
known as people-centric web, and participative web (Aghaei et al., 2012). The characteristics
of the Web 2.0 include flexible web design, creative reuse, updates,
collaborative content creation and modification (Aghaei et al., 2012). The main
technologies and services of Web 2.0 include blogs, really simple syndication
(RSS), wikis, mashups, tags, folksonomy, and tag clouds (Aghaei et al., 2012). In Web 2.0, three
main approaches are used to develop such interactive applications: Asynchronous JavaScript and XML (AJAX), Flex,
and the Google Web Toolkit (Aghaei et al., 2012). Web 2.0 has three major limitations (Choudhury, 2014). The first
limitation is the constant iteration cycle of modification and update to
services. The second limitation is reflected in the ethical issues concerning
the development and the usage of Web 2.0 (Choudhury, 2014). The third
limitation is represented in the interconnectivity and knowledge sharing
between platforms across community boundaries which are still limited (Choudhury, 2014).
The third generation of the Web is Web
3.0, which was introduced in 2006 as “semantic
web” (Aghaei et al., 2012; Kambil,
2008; Patel, 2013)
or “executable Web” (Choudhury, 2014). Web 3.0 includes two
main platforms of the semantic technologies and the social computing (Aghaei et al., 2012; Patel,
2013).
The semantic technologies open standards which can be applied on the top of the
web, while the social computing allows the cooperation of the machine and the
organization of a large number of social
web communities (Aghaei et al., 2012). With Web 3.0, the
data can be linked, integrated and analyzed using data sets to obtain a new stream of information (Aghaei et al., 2012). With Web 3.0, the
data management is improved, the accessibility of the mobile internet is supported, the creativity and innovation are
simulated, customers’ satisfaction is enhanced, and the collaboration of the
social web is organized (Aghaei et al., 2012). The Semantic Web emerged to overcome the problem of the
current web which is represented in the “web of documents” (Aghaei et al., 2012). The Semantic Web is
defined as the “Web of Data,” where primary
objects are regarded as things which can be linked (Aghaei et al., 2012;
Choudhury, 2014; Patel, 2013). The World-Wide Web Consortium (W3C) team developed
Resource Description Framework (RDF) to provide a framework to describe and
exchange meta-data on the Web (Devlic & Jezic, 2005), to improve and extend, and standardize
the existing systems, tools and languages (Aghaei et al., 2012; Choudhury, 2014;
Patel, 2013).
In 2007, Berners-Lee developed rules
known as Linked Data principles to publish and connect data on the web for
Semantic Web (Aghaei et al., 2012). These rules include
URI, HTTP, RDF as follows:
URIs should be used as names for things,
HTTP URIs should be used to look up
those names,
URI should be used to provide useful
information using the standards RDF, SPARQL, and
Links to other URIs should be
included to discover more things (Aghaei et al., 2012).
Web 4.0 is the fourth generation of
the Web and is known as the Web of Integration (Aghaei et al., 2012), and is considered as
an “Ultra-Intelligent Electronic Agent,” “symbiotic web” and “ubiquitous web” (Patel, 2013). In Web 4.0 machines
can read the contents of the web, and
execute and decide the first execution to load the website fast with a high
quality and superior performance and develop more commanding interfaces (Patel, 2013). Web 4.0 will be
read-write and concurrency web. The
first Web 4.0 was the consumer electronic where consumers are recognized, and
personalized advice is offered as the case with Amazon (Patel, 2013). The migration of
online functionality into the physical world is regarded to be the most
critical development of Web 4.0 (Patel, 2013).
Web 5.0 is the fifth generation of
the Web and is in progress, and there is no exact definition of how it
would be (Patel, 2013). However, Web 5.0 can be regarded as decentralization
of the “Symbionet Web,” whose servers can use a part of “memory and calculation
power” of each interconnected SmartCommunicator (SC) such as smartphones, or
tablets, in order to calculate billions of data to develop 3D world, and feed the
Artificial Intelligence (Patel, 2013).
From the computing platform perspective, each generation of the Web has its characteristics. The underlying computing platform of the Web 1.0 was the traditional client-server based distributed computing (Aghaei et al., 2012; Patel, 2013).
The computing platform of Web 2.0 is represented in grid, cluster, and cloud
computing (Foster, Zhao, Raicu, & Lu, 2008). The grid computing emerged from
distributed computing and parallel processing technologies (Dubitzky, 2008). Grid computing supports various
kinds of applications ranging from high-performance
computing (HPC) to high throughput computing (HTC) (Foster et al., 2008). With grid computing resources are shared to provide advantages such as overcoming
of bottlenecks face by much large-scale
application, the adaption to unexpected failure, integration of heterogeneous
resources and systems, and providing cost/performance ratio making
high-performance computing affordable (Dubitzky, 2008). However, grid computing faced challenges resulted from the complexity of the heterogeneity of the
underlying software and hardware resources, decentralized control, techniques
to deal with the faults and loss of resources, security, and privacy (Dubitzky, 2008).
As indicated in (Ji, Li, Qiu, Awada, & Li, 2012), with the success of Web 2.0, the needs to store and analyze the growing
data, such as search logs, crawled web content, and click streams have been increased.
The Cloud computing evolved out of grid computing and relied on grid
computing as its backbone and infrastructure support (Foster et al., 2008).
The computing platform for Web 3.0 is reflected in the mobile and sensor-based
applications which analyze the location and the context-aware techniques for
collecting, processing, analyzing and visualizing such large-scale (Chen, Chiang, & Storey, 2012).
The computing platform for Web 4.0 is reflected in the increased real-time integration between users and the virtual worlds and objects they interact with (Kambil, 2008), while the computing platform for Web 5.0 is reflected in the decentralized smart communicator as indicated in (Patel, 2013).
In summary, HTTP, HTML, and URI have been the core of the Web since Web 1.0, HTTP is used for communication, HTML is used for web pages, and URI is used for identifying web objects such as web pages or frames. Although additional Web versions are merged from Web 2.0 to Web 5.0, these protocols are still the core, which is preserved by the new generation of the Web. The new generations of the Web are emerged to address some of the limitation of the previous generation and advance the capabilities of the Web.
References
Aghaei, S., Nematbakhsh, M. A.,
& Farsani, H. K. (2012). Evolution of the world wide web: From WEB 1.0 TO
WEB 4.0. International Journal of Web
& Semantic Technology, 3(1), 1.
Berners-Lee, T.
(1996). WWW: Past, present, and future. Computer,
29(10), 69-77.
Berners-Lee, T.,
Hendler, J., & Lassila, O. (2001). The semantic web.
Chen, H., Chiang,
R. H. L., & Storey, V. C. (2012). Business Intelligence and Analytics: From
Big Data to Big Impact. MIS quarterly, 36(4),
1165-1188.
Choudhury, N.
(2014). World Wide Web and its journey from web 1.0 to web 4.0.
Devlic, A., &
Jezic, G. (2005). Location-aware
information services using user profile matching.
Dubitzky, W.
(2008). Data Mining in Grid Computing
Environments: John Wiley & Sons.
Foster, I., Zhao,
Y., Raicu, I., & Lu, S. (2008). Cloud
Computing and Grid Computing 360-Degree Compared. Paper presented at the
2008 Grid Computing Environments Workshop.
Jacobs, I., &
Walsh, N. (2004). Architecture of the world wide web.
Ji, C., Li, Y.,
Qiu, W., Awada, U., & Li, K. (2012). Big
Data Processing in Cloud Computing Environments. Paper presented at the
2012 12th International Symposium on Pervasive Systems, Algorithms and
Networks.
Kambil, A.
(2008). What is your Web 5.0 strategy? Journal
of business strategy, 29(6), 56-58.
Patel,
K. (2013). Incremental journey for World Wide Web: introduced with Web 1.0 to
recent Web 5.0–a survey paper.
Think and Knowledge Tank 