Sunday, December 20, 2015

IT (4): Databases (CouchDB, Storm))........

As big data arrives, databases have become a critical part of the data storage and mining.
Common databases: CouchDB,
Database management systems: Relational Database Management System (RDBMS), NoSQL

RDBMS (table based) : MS SQL server, Oracle, IBM DB2, MySQL
NoSQL (key-value, colun and documents based): Cassandra, DynamoDB, BigTable, MongoDB, CouchDB
These two systems vary in their architecture, data handling. NoSQL
Creation, deletion, updating, installation are part of database technology.
When Oracle is running in your environment, you need some DBA tasks, like exporting data from the Oracle database. Oracle’s exp utility, it is very easy to dump data from database.
Connect to DB in exp utility
Export (full database; one or more specific Schemas/Users; one or more specific Tables; all objects in a Tablespac
ls -l expdat.dmp

Its written in Erlang language and now operated by Apache software
Its a document store database of NoSQL
JSON to store data, javascript to mine data, HTTP for API
Communication with HTTP protocol using GET (get a particular item), HEAD (to find only header of an item), POST (to upload data), PUT (create new items), DELETE (remove items), COPY (copy items)
Interaction with CouchDB with curl utility and Futon to create, delete and update databases and documents
#Communicating with the database
#Get list of databases
curl -X GET
#Create a new  database
curl -X PUT
#To check if the database is created
curl -X GET
#To get information of the newly created database

curl -X GET
Futon  is web-based interface of CouchDB for database and documents handling.
#To create database using Futon
#Creating a Document using curl
#Document is Field-value separated by colon closed in braces
curl -X PUT name/"id" -d ' { document} '
curl -X PUT name/"01" -d ' 
Name : Carl,   Age : 32,    Job : Artist,    City: Paris} '

#Response of the database as 'ok', 'id', 'rev'
#Verifying the Document using curl
curl -X GET name/01
Apache Storm is a free and open source distributed realtime computation system. 
Storm makes it easy to reliably process unbounded streams of data.
Storm can be used with any programming language.
Its usage include realtime analytics, online machine learning, continuous computation.

Its fast as over a million tuples can be processed per second per node. 

No comments:

Post a Comment