Big Data

Explore a career in Big Data.

Big data is a field that treats ways to analyze, systematically extract information from, or otherwise deal with data sets that are too large or complex to be dealt with by traditional data-processing application software.

• Sample Industry Definitions, Job Titles & Project Areas
  • Data Analyst – primary focus of analysis and problem solving relates to data, types of data, and relationships among data elements within a business system or IT system.
  • Database Administrator – ensure databases run efficiently, uses specialized software to store and organize data and make sure that data is available to users and secure from unauthorized access.
  • Database Developer – create and implement computer databases, determine the best database management systems for a particular client, as well as test database programs for efficiency and performance, troubleshoot and correct problems.
  • Data Modeler – systems analysts that design computer databases that translate complex business data into usable computer systems, they work with data architects to design databases that meet organizational needs using conceptual, logical, and physical data models.
  • Data Scientist – collects, analyzes and interprets large amounts of data to identify ways to help a business improve operations an gain a competitive edge over rivals.
  • Research Scientists – these individuals are experts in multiple AI disciplines, including applied mathematics, machine learning, deep learning, and computational statistics.
  • Business Intelligence Analyst – analyzes data that is used by a business or organization. Data used in business intelligence generally supports decision-making, the analyst works with this kind of data to maximize its utility.
  • Database Manager – provides database applications by developing the architecture, programming, troubleshooting, and security; maintains database results by setting and enforcing standards and controls; upgrades hardware and software by assessing transaction processing
  • Data Warehouse Manager – manages daily activities of team responsible for design, implementation, maintenance, and support of data warehouse systems and related data marts; oversees data design and the creation of database architecture and data repositories.
  • Data Architect – designs, creates, deploys and manages an organization’s data architecture; defines how the data will be stored, consumed, integrated and managed by different data entities and IT systems.
  • Big Data Engineer – creates and manages a company’s Big Data infrastructure and tools, and is someone that knows how to get results from vast amounts of data quickly; builds what the big data solutions architect has designed; develop, maintain, test and evaluate big data solutions within organizations.
  • Big Data Analyst – reviews, analyzes, and reports on big data stored and maintained by an organization; responsible for utilizing data analytics and customer relationship management (CRM) software
  • to evaluate an organization’s technical performance and providing recommendations on system enhancements
  • Data Visualization Developer/Analyst – utilizes visualization tools and software to communicate information to clients or for their own company by presenting large amounts of information in ways that are universally understandable or easy to interpret and spot patterns, trends, and correlations.
  • Machine Learning Engineer – computer programmers that develop machines and systems that can learn and apply knowledge without specific direction (i.e.: using building towards artificial intelligence). They will create programs that will enable machines to take actions without being specifically directed to perform those tasks.
  • Predictive Analytics – the practice of extracting information from existing data sets in order to determine patterns and predict future outcomes and trends; forecasts what might happen in the future with an acceptable level of reliability, and includes what-if scenarios and risk assessment
  • Prescriptive Analytics – uses optimization and simulation algorithms to advise on possible outcomes; attempt to quantify the effect of future decisions in order to advise on possible outcomes before the decisions are actually made.
  • Descriptive Analytics – describes or summarizes raw data and make something that is interpretable by humans; specifically understanding past behaviors and how they might influence the future.
• Industries that use Big Data
  • Business
  • E-commerce
  • Finance
  • Government
  • Healthcare
  • Science
  • Social Networking
  • Telecommunications
  • Utilities
  • Politics
  • Smart Meters on Appliances
• Defining Characteristics
  • Volume (storage, identifying relevant data)
  • Velocity (the pace at which data is produced, collected, and processed)
  • Veracity (quality of data)
  • Variety (form, structure, sorting)
• Skills / Tools
  SPSS, SAS, R, SQL, Apache Hadoop, Pig, Hive, Storm, NoSQL, Statistics, Python, Data Visualization.
• Companies to Consider
  IBM, Amazon, Microsoft, Facebook, Oracle, Google, Apple, Intel, HCL Technologies, Wipro Technologies, Verizon, Visa, Boeing, SA.