Blockchain and Bitcoin – Industry Insights & Reference Architectures…

Distributed Ledger technology (DLT) and applications built for DLT’s – such as cryptocurrencies – are arguably the hottest topics in tech. This post summarizes seven key blogs on the topic of Blockchain and Bitcoin published at VamsiTalksTech.com. It aims to serve as a handy guide for business and technology audiences tasked with understanding and implementing this groundbreaking technology.

Image Credit – DCEBrief

Introduction…

We have been discussing the capabilities of Blockchain and Bitcoin for quite some time on this blog. The impact of Blockchain on many industries is now clearly apparent. But can the DLT movement enable business efficiency and profitability?

# 1 – Introduction to Bitcoin –

Bitcoin (BTC) is truly the first decentralized, peer to peer, high secure and purely digital currency. Bitcoin & it’s other cousins such as Ether and AltCoins now regularly get widespread (& mostly positive) notice by a range of industry actors- ranging from Consumers, Banking Institutions, Retailers & Regulators. Riding on the real pathbreaker – the Blockchain, the cryptocurrency movement will help drive democratization in the financial industry and society at large in the years to come. This blog post discusses BTC from a business standpoint.

Bitcoin (BTC) Ushers in the Future of Finance..(1/5)

# 2 – The Architecture of Bitcoin –

This post discusses the technical architecture of Bitcoin.

The Architecture of Bitcoin..(2/5)

# 3 – Introduction to Blockchain –

The term Blockchain is derived from a design pattern that describes a chain of data blocks that map to individual transactions. Each transaction that is conducted in the real world (e.g a Bitcoin wire transfer) results in the creation of new blocks in the chain. The new blocks so created are done so by calculating a cryptographic hash function of its previous block thus constructing a chain of blocks – hence the name. This post introduces the business potential of Blockchain to the reader.

The immense potential of the Blockchain..(3/5)

# 4 – The Reference Architecture of the Blockchain –

This post discusses the technical architecture of Blockchain.

The Architecture of Blockchain..(4/5)

# 5 – How Blockchain will lead to Industry disruption –

Blockchain lies at the heart of the Bitcoin implementation & is easily the most influential part of the BTC platform ecosystem. Blockchain is thus both a technology platform and a design pattern for building global scale industry applications that make all of the above possible. Its design makes it possible to be used as a platform for digital currency also enable it to indelibly record any kind of transaction – be it a currency, or, a medical record, or, supply chain data, or, a document etc into it.

How the Blockchain will lead disruption across industry..(5/5)

# 6 – What Blockchain can do for the Internet of Things (IoT) –

Blockchain can enable & augment a variety of application scenarios and use-cases for the IoT. No longer are such possibilities too futuristic – as we discuss in this post.

What Blockchain can do for The Internet Of Things..

# 7 – Key Considerations in Adapting the Blockchain for the Enterprise –

With advances in various Blockchain based DLTs (distributed ledger technology) platforms such as HyperLedger & Etherium et al, enterprises have begun to take baby steps to adapt the Blockchain (BC) to industrial scale applications. This post discusses some of the stumbling blocks the author has witnessed enterprises are running into as they look to adopt Blockchain based Distributed Ledger Technology in real-world applications.

Blockchain For the Enterprise: Key Considerations..

Conclusion..

The true disruption of Blockchain based distributed ledgers will in moving companies to an operating model where they leave behind siloed and stovepiped business processes, to the next generation of distributed business processes predicated on a seamless global platform. The DLT based platform will enable the easy transaction, exchange, and contraction of digital assets. However, before enterprises rush in, they need to perform an adequate degree of due diligence to avoid some of the pitfalls we have highlighted above.

Blockchain For the Enterprise: Key Considerations..

With advances in various Blockchain based DLTs (distributed ledger technology) platforms such as HyperLedger & Etherium et al, enterprises have begun to take baby steps to adapt the Blockchain (BC) to industrial scale applications. This post discusses some of the stumbling blocks the author has witnessed enterprises are running into as they look to get started on this journey. 

Image Credit – Blockchain Technologies

Blockchain meets the Enterprise…

The Blockchain is a system & architectural design pattern for recording (immutable) transactions while providing an unalterable historical audit trail. This approach (proven with the hugely successful Bitcoin) guarantees a high degree of security, transparency, and anonymity for distributed applications purpose built for it. Bitcoin is but the first application of this ground breaking design pattern.

Due to its origins in the Bitcoin ecosystem, there has been a high degree association of the Blockchain with the cryptocurrency movement. However, a wide range of potential enterprise applications has been identified in industries such as financial services, healthcare, manufacturing, and retail etc – as depicted in the below illustration.

The Evolution of Blockchain from a crypto-anarchist platform to a platform for distributed business applications.

Last year, we took an in-depth look into the business potential of the Blockchain design pattern at the below post.

The immense potential of the Blockchain..(3/5)

We can then define the Enterprise Blockchain as “a highly secure, resilient, algorithmic & accurate globally distributed ledger (or global database or the biggest filesystem or the largest spreadsheet) that provides an infrastructure pattern to build multiple types of applications that help companies (across every vertical), consumers and markets discover new business models, transact, trade & exchange information & assets.”

While some early deployments and initial standards making activity have been seen in financial services and healthcare, it is also finding significant adoption in optimizing internal operations for globally diversified conglomerates. For instance, tech major IBM claims to host one of the largest blockchain enterprise deployments. The application known as IGF provides working capital to about to 4,000+ customers, distributors, and partners. IBM uses its blockchain platform to manage disputes in the $48 billion IGF program. [1]. The near linear scalability of the blockchain ensures that the IGF can gradually increase the number of members participating in the network.

Image Credit – Mark Morris and IBM [3]
In particular, the Financial Services Industry has had several bodies aiming to create standards around use cases such as consumer and correspondent banking payments and around the trade lifecycle. Some examples of these are R3 Corda, HyperLedger, and Ethereum. However, there is still a large amount of technology innovation, adoption and ecosystem development that needs to happen before the technology is consumable by your everyday bank or manufacturer or insurer.

The Four Modes of Blockchain Adoption in the Enterprise…

There are certain criteria that need to be met for a business process to benefit from a distributed ledger. First off, the business process should comprise various actors (both internal and external to the organization), secondly, there should be no reason to have a central authority or middleman to verify daily transactions except when disputes arise. Third, the process should call for strict audit trail as well as transaction immutability. The assets stored on the blockchain can really be anything – data, contracts or transactions etc.

At a high level, there are four modes of adoption, or, ways in which a BC technology can make its way into an enterprise –

  • Organic Proof Of Concept’s (POC) – These are driven by innovation groups inside the company tasked with exploring the latest technology advances. Oftentimes, these are technology-driven initiatives in search of a business problem. The approach works like this – management targets specific areas in technology where the firm needs to develop capabilities around. The innovation team works on defining an appropriate technical approach, reference stack & architecture (in this case for applications that have been determined to be suitable to be POC’d on a DLT) et al. The risk in this approach is that much of the best practices, learnings etc from other organizations, vendors, and solution providers are not leveraged.
  • Participation in Industry Consortia – A consortium is a group of companies engaged in a similar business task. These kinds of initiatives are being driven by like minded enterprises banding together (within specific sectors such as financial services, insurance, and healthcare) to define common use-cases that can benefit from sector specific common standards from a DLT standpoint & the ensuing network effects. Consortiums tend to mitigate risk both from a business and a cost standpoint as several companies typically band together to explore the technology. However, these can be difficult to pull off many a time due to competitive and cultural reasons.
  • In many cases, Regulators are pushing industry leaders to look into use cases (such as Risk Management, BackOffice Processes, and Fraud Detection) which can benefit from adopting distributed ledger technology (DLT).
  • Partnerships with Blockchain start-ups – These arrangements enable the (slow to move) incumbent market leading enterprises to partner with the brightest entrepreneurial minds in the BC world who are building path-breaking applications which will upend business models. The focus of such efforts has been to identify a set of use-cases & technology approaches that would immensely help the organization from applying BC technology to their internal and external business challenges. The advantages of this approach are that the skills shortage when established companies tackle immature technology projects can be ameliorated by working with younger organizations.

Having noted all this, the majority of proof of concepts driven out of enterprises are failing or performing suboptimally.

I feel that this is due to various reasons some of which we will discuss below. Point to be noted is that we are assuming that there is strong buy-in around BC and DLT at the highest levels of the organization. Scepticism about this proven design pattern and overcoming it is quite another topic altogether.

The Key Considerations for a Successful Enterprise Blockchain or Distributed Ledger  (DLT)…

CONSIDERATION #1 – Targeting the right business use case for the DLT…

As we saw in the above sections, the use cases identified for DLT need to reflect a few foundational themes -non-reliance on a middleman, a business process supporting a truly distributed deployment, building trust among a large number of actors/counterparties, ability to support distributed consensus, and transparency. Due to its flat, peer to peer nature – Blockchain/DLT conclusively eliminates the need for any middleman.It is important that a target use case be realistic from both a functional requirement standpoint as well as from a business process understanding. The majority of enterprise applications can do perfectly well with a centralized database and applying DLT technology to them can cause projects to fail.

CONSIDERATION #2 – The Revenge of the Non Functional requirements…

Generally speaking, the current state of DLT platforms is that they fall short in a few key areas that enterprises usually take for granted in other platforms such as Cloud Computing, Middleware, Data platforms etc. These include key areas such as data privacy, transaction throughput, high speed of performance etc. If one recalls, the community Blockchain (that Bitcoin was built on) prioritized anonymity over privacy. This can sometimes be undesirable in areas such as payments processing or healthcare where the identitiy of consumers is governed by strict KYC (Know Your Customer) mandates. Thus, from an industry standpoint most DLT platforms are 24 months or so away from coming up to par in these areas in a manner that enterprises can leverage them.

Some of the other requirements, such as performance and scalability, are sometimes not directly tied to business features but lack of support for them can stymie any ambitious intended use of the technology. For instance, a key requirement in payments processing and supplier management is the ability for the platform to process a large number of transactions per second. Most DLT’s can only process around ten transactions per second on a permissionless network. This is far far from the ideal throughput needed in use-cases such as payments processing, IoT etc.

The good news is that the DLT community are acutely aware of enhancements that need to be done to the underlying platforms (e.g reduced block size etc) to increase throughput but these changes will take time to make their way into the mass market given the rigorous engineering work that needs to happen.

CONSIDERATION #3 – Neglecting Enterprise Integration Requirements…

The Blockchain/DLT is not a data management paradigm. This is important for adopters to understand. Also, there currently exist very few standards and guidance on integrating distributed applications (Dapps) custom built for DLTs with underlying enterprise assets. These assets include enterprise middleware stacks, identity management platforms, corporate security systems, application data silos, BPM (Business Process Management) and Robotic Process Automation systems etc. For the BC to work for any business capability and as a complete business solution, necessary integration must be provided with a reasonable number of backend systems that influence the business cases- most such integration is sorely lacking. Interoperability is still in its infancy despite vendor claims.

CONSIDERATION #4 – Understand that Smart Contracts are still in their infancy…

The blockchain introduces the important notion of programmable digital instruments or contracts. An important illustration of the possibilities of blockchain is this concept of a “Smart contract”. Instead of static data objects that are inserted into the distributed ledger, a Smart Contract is a program that can perform the generation of downstream actions when appropriate conditions are met. They only become immutable once accepted into the ledger. Business rules are embedded in a contract that can automatically trigger based on certain conditions being met. E.g. a credit pre-qualification or assets transferred after a payment is made or after legal approval is provided etc.

Smart Contracts are being spoken about as the key functionality for any DLT platform based on Blockchain. While this hype is justified in some sense, it should be noted that smart contracts are again not standards based across major DLT platforms. Which means that they’re not auditable & verifiable across both local and global jurisdictions or when companies use different underlying commercial DLTs. The technology will evolve over the next few years but it is still very early days to run large scale production grade applications that leverage Smart Contracts.

CONSIDERATION #5 – SECURITY and DATA PRIVACY CONCERNS…

The promise of the original blockchain platform which ran Bitcoin was very simple. It provided a truly secure, trustable and immutable record on which any digital asset could be run. Parties using the system were all in a permissionless mode which meant that their identities were hidden from one another and from any central authority. While this may work for Bitcoin like projects, the vast majority of industry verticals will need strong legal agreements and membership management capabilities which follow them. Accordingly, these platforms will need to be permission-ed.

CONSIDERATION #6 – Blockchain Implementations need to be treated as AN INTEGRAL part of Digital Transformation…

Blockchain as a technology definitely sounds way more exotic than Digital projects which have all the idea currency at the moment. However, an important way to visualize the organizational BC is that it provides an environment of instantaneous collaboration with business partners and customers. That is a core theme of Digital Transformation as one can appreciate. Accordingly, Blockchain/DLT proof of concepts themselves should be centrally funded & governed, skills should be grown in this area from both a development, administration and project management standpoint. Projects should be tracked using fair business metrics and appropriate governance mechanisms instituted as with any other digital initiative.

Conclusion…

Surely, Blockchain based distributed ledgers are going to usher in the next generation of distributed business processes. These will enable the easy transaction, exchange, and contraction of digital assets. However, before enterprises rush in, they need to perform an adequate degree of due diligence to avoid some of the pitfalls highlighted above.

References…

[1] IDC Insights – “IBM Wants to Make 2017 the Year of Blockchain Enterprise Deployment” https://www.idc.com/getdoc.jsp?containerId=EMEA42454617

[2] Coindesk – “Spanish Bank BBVA Joins Hyperledger Blockchain Project” –  https://www.coindesk.com/bbva-hyperledger-blockchain-project/

[3] “Blockchain: Supply Chain Dispute Resolution Killer Solution” – Mark Morris

https://www.linkedin.com/pulse/blockchain-supply-chain-dispute-resolution-killer-solution-morris

A Digital Reference Architecture for the Industrial Internet Of Things (IIoT)..

A few weeks ago on the invitation of DZone Magazine, I jointly authored a Big Data Reference Architecture along with my friend & collaborator, Tim Spann (https://www.linkedin.com/in/timothyspann/). Tim & I distilled our experience working on IIoT projects to propose an industrial strength digital architecture. It brings together several technology themes – Big Data , Cyber Security, Cognitive Applications, Business Process Management and Data Science. Our goal is to discuss a best in class architecture that enables flexible deployment for new IIoT capabilities allowing enterprises to build digital applications. The abridged article was featured in the new DZone Guide to Big Data: Data Science & Advanced Analytics which can be downloaded at  https://dzone.com/guides/big-data-data-science-and-advanced-analytics

How the Internet Of Things (IoT) leads to the Digital Mesh..

The Internet of Things (IoT) has become one of the four top hyped up technology paradigms affecting the world of business. The other usual suspects being Big Data, AI/Machine Learning & Blockchain. Cisco predicts that the IOT is expected to impact about 25 billion connected things by 2020 and affect about $2 trillion of economic value globally across a diverse range of verticals. These devices are not just consumer oriented devices such as smartphones and home monitoring systems but dedicated industry objects such as sensors, actuators, engines etc.

The interesting angle to all this is the fact that autonomous devices are already beginning to communicate with one another using IP based protocols. They largely exchanging state & control information around various variables. With the growth of computational power on these devices, we are not far off from their sending over more granular and interesting streaming data – about their environment, performance and business operations – all of which will enable a higher degree of insightful analytics to be performed on the data. Gartner Research has termed this interconnected world where decision making & manufacturing optimization can occur via IoT as the “Digital Mesh“.

The evolution of technological innovation in areas such as Big Data, Predictive Analytics and Cloud Computing now enables the integration and analysis of massive amounts of device data at scale while performing a range of analytics and business process workflows on the data.

Image Credit – Sparkling Logic

According to Gartner, the Digital Mesh will thus lead to an interconnected data information deluge powered by the continuous data from these streams. These streams will encompasses classical IoT endpoints (sensors, field devices, actuators etc) sending data in a variety of formats –  text, audio, video & social data streams – along with new endpoints in areas as diverse as Industrial Automation, Remote Healthcare, Public Transportation, Connected Cars, Home Automation etc. These intelligent devices will increasingly begin communicating with their environments in a manner that will encourage collaboration in a range of business scenarios. The industrial cousin of IoT is the Industrial Internet of Things (IIIoT).

Defining the Industrial Internet Of Things (IIoT)

The Industrial Internet of Things (IIoT) can be defined as a ecosystem of capabilities that interconnects machines, personnel and processes to optimize the industrial lifecycle.  The foundational technologies that IIoT leverages are Smart Assets, Big Data, Realtime Analytics, Enterprise Automation and Cloud based services.

The primary industries impacted the most by the IIoT will include Industrial Manufacturing, the Utility industry, Energy, Automotive, Transportation, Telecom & Insurance.

According to Markets and Markets, the annual worldwide Industrial IoT market is projected to exceed $319 billion in 2020, which represents an 8% a compound annual growth rate (CAGR). The top four segments are projected to be manufacturing, energy and utilities, auto & transportation and healthcare.[1]

Architectural Challenges for Industrial IoT versus Consumer IoT..

Consumer based IoT applications generally receive the lion’s share of media attention. However the ability of industrial devices (such as sensors) to send ever more richer data about their operating environment and performance characteristics is driving a move to Digitization and Automation across a range of industrial manufacturing.

Thus, there are four distinct challenges that we need to account for in an Industrial IOT scenario as compared to Consumer IoT.

  1. The IIoT needs Robust Architectures that are able to handle millions of device telemetry messages per second. The architecture needs to take into account that all kinds of devices operating in environments ranging from the constrained to
  2. IIoT also calls for the highest degrees of Infrastructure and Application reliability across the stack. For instance, a lost message or dropped messages in a healthcare or a connected car scenario may mean life or death for a patient, or, an accident.
  3. An ability to integrate seamlessly with existing Information Systems. Lets be clear, these new age IIOT architectures need to augment existing systems such as Manufacturing Execution Systems (MES) or Traffic Management Systems. In Manufacturing, MES systems continually improve the product lifecycle and perform better resource scheduling and utilization. This integration helps these systems leverage the digital intelligence and insights across (potentially) millions of devices across complex areas of operation.
  4. An ability to incorporate richer kinds of analytics than has been possible before that provide a great degree of context. This ability to reason around context is what provides an ability to design new business models which cannot be currently imagined due to lack of agility in the data and analytics space.

What will IIoT based Digital Applications look like..

Digital Applications are being designed for specific device endpoints across industries. While the underlying mechanisms and business models differ from industry to industry, all of these use predictive analytics based on a combination of real time data processing & data science algorithms. These techniques extract insights from streaming data to provide digital services on existing toolchains, provide value added customer service, predict device performance & failures, improve operational metrics etc.

Examples abound. For instance, a great example in manufacturing is the notion of a Digital Twin which Gartner called out last year. A Digital twin is a software personification of an Intelligent device or system.  It forms a bridge between the real world and the digital world. In the manufacturing industry, digital twins can be setup to function as proxies of Things like sensors and gauges, coordinate measuring machines, vision systems, and white light scanning. This data is sent over a cloud based system where it is combined with historical data to better maintain the physical system.

The wealth of data being gathered on the shop floor will ensure that Digital twins will be used to reduce costs and increase innovation. Thus, in global manufacturing – Data science will soon make it’s way into the shop floor to enable the collection of insights from these software proxies. We covered the phenomenon of Servitization in manufacturing in a previous blogpost.

In the Retail industry, an ability to detect a customer’s location in realtime and combining that information with their historical buying patterns can drive real time promotions and an ability to dynamically price retail goods.

Solution Requirements for an IIoT Architecture..

At a high level, the IIoT reference architecture should support six broad solution areas-

  1. Device Discovery – Discovering a range of devices (and their details)  on the Digital Mesh for an organization within and outside the firewall perimeter
  2. Performing Remote Lifecycle Configuration of these devices ranging from startup to modification to monitoring to shut down
  3. Performing Deep Security level introspection to ensure the patch levels etc are adequate
  4. Creating Business workflows on the Digital Mesh. We will do this by marrying these devices to enterprise information systems (EISs)
  5. Performing Business oriented Predictive Analytics on these devices, this is critical to 
  6. On a futuristic basis, support optional integration with the Blockchain to support a distributed organizational ledger that can coordinate activity across all global areas that an enterprise operates in.

Building Blocks of the Architecture

Listed below are the foundational blocks of our reference architecture. Though the requirements will vary across industries, an organization can reasonably standardize on a number of foundational components as depicted below and then incrementally augment them as the interactions between different components increase based on business requirements.

Our reference architecture includes the following major building blocks –

  • Device Layer
  • Device Integration Layer
  • Data & Middleware Tier
  • Digital Application Layer

It also includes the following cross cutting concerns which span across the above layers –

  • Device and Data Security
  • Business Process Management
  • Service Management
  • UX Design
  • Data Governance – Provenance, Auditing, Logging

The next section provides a brief overview of the reference architecture’s components at a logical level.

A Big Data Reference Architecture for the Industrial Internet depicting multiple functional layers

Device Layer – 

The first requirement of IIIoT implementations is to support connectivity from the Things themselves or the Device layer depicted at the bottom. The Device layer includes a whole range of sensors, actuators, smartphones, gateways and industrial equipment etc. The ability to connect with devices and edge devices like routers, smart gateways using a variety of protocols is key. These network protocols include Ethernet, WiFi, and Cellular which can all directly connect to the internet. Other protocols that need a gateway device to connect include Bluetooth, RFID, NFC, Zigbee et al. Devices can connect directly with the data ingest layer shown above but it is preferred that they connect via a gateway which can perform a range of edge processing.

This is important from a business standpoint for instance, in certain verticals like healthcare and financial services, there exist stringent regulations that govern when certain identifying data elements (e.g. video feeds) can leave the premises of a hospital or bank etc. A gateway cannot just perform intelligent edge processing but also can connect thousands of device endpoints and facilitate bidirectional communication with the core IIoT architecture. 

The ideal tool for these constantly evolving devices, metadata, protocols, data formats and types is Apache NiFi.  These agents will send the data to an Apache NiFi gateway or directly into an enterprise Apache NiFi cluster in the cloud or on-premise.

Apache NiFi Eases Dataflow Management & Accelerates Time to Analytics In Banking (2/3)..

 A subproject of Apache NiFi – MiNiFi provides a complementary data collection approach that supplements the core tenets of NiFi in dataflow management. However due to its small footprint and low resource consumption, is well suited to handle dataflow from sensors and other IOT devices. It provides central management of agents while providing full chain of custody information on the flows themselves.

For remote locations, more powerful devices like the Arrow BeagleBone Black Industrial and MyPi Industrial, it is very simple to run a tiny Java or C++ MiNiFi agent for secure connectivity needs.

The data sent by the device endpoints are then modeled into an appropriate domain representation based on the actual content of the messages. The data sent over also includes metadata around the message. A canonical model can optionally be developed (based on the actual business domain) which can support a variety of applications from a business intelligence standpoint.

 Apache NiFi supports the flexibility of ingesting changing file formats, sizes, data types and schemas. The devices themselves can send a range of feeds in different formats. E.g. XML now and based on upgraded capabilities – richer JSON tomorrow. NiFi supports ingesting any file type that the devices or the gateways may send.  Once the messages are received by Apache NiFi, they are enveloped in security with every touch to each flow file controlled, secured and audited.   NiFi flows also provide full data provenance for each file, packet or chunk of data sent through the system.  NiFi can work with specific schemas if there are special requirements for file types, but it can also work with unstructured or semi structured data just as well.  From a scalability standpoint, NiFi can ingest 50,000 streams concurrently on a zero-master shared nothing cluster that horizontally scales via easy administration with Apache Ambari.

Data and Middleware Layer – 

The IIIoT Architecture recommends a Big Data platform with native message oriented middleware (MOM) capabilities to ingest device mesh data. This layer will also process device data in such a fashion – batch or real-time – as the business needs demand.

Application protocols such as AMQP, MQTT, CoAP, WebSockets etc are all deployed by many device gateways to communicate application specific messages.  The reason for recommending a Big Data/NoSQL dominated data architecture for IIOT is quite simple. These systems provide Schema on Read which is an innovative data handling technique. In this model, a format or schema is applied to data as it is accessed from a storage location as opposed to doing the same while it is ingested. From an IIOT standpoint, one must not just deal with the data itself but also metadata such as timestamps, device id, other firmware data such as software version, device manufactured data etc. The data sent from the device layer will consist of time series data and individual measurements.

The IIoT data stream can thus be visualized as a constantly running data pump which is handled by a Big Data pipeline takes the raw telemetry data from the gateways, decides which ones are of interest and discards the ones not deemed significant from a business standpoint.  Apache NiFi is your gateway and gate keeper.   It ingests the raw data, manages the flow of thousands of producers and consumers, does basic data enrichment, sentiment analysis in stream, aggregation, splitting, schema translation, format conversion and other initial steps to prepare the data. It does that all with a user-friendly web UI and easily extendible architecture.  It will then send raw or processed data to Kafka for further processing by Apache Storm, Apache Spark or other consumers.  Apache Storm is a distributed real-time computation engine that reliably processes unbounded streams of data.  Storm excels at handling complex streams of data that require windowing and other complex event processing. While Storm processes stream data at scale, Apache Kafka distributes messages at scale. Kafka is a distributed pub-sub real-time messaging system that provides strong durability and fault tolerance guarantees. NiFi, Storm and Kafka naturally complement each other, and their powerful cooperation enables real-time streaming analytics for fast-moving big data. All the stream processing is handled by NiFi-Storm-Kafka combination.  

Apache Nifi, Storm and Kafka integrate very closely to manage streaming dataflows.

 

Appropriate logic is built into the higher layers to support device identification, ID lookup, secure authentication and transformation of the data. This layer will process data (cleanse, transform, apply a canonical representation) to support Business Automation (BPM), BI (business intelligence) and visualization for a variety of consumers. The data ingest layer will also providing notification and alerts via Apache NiFi.

Here are some typical uses for this event processing pipeline:

a. Real-time data filtering and pattern matching

b. Enrichment based on business context

c. Real-time analytics such as KPIs, complex event processing etc

d. Predictive Analytics

e. Business workflow with decision nodes and human task nodes

Digital Application Tier – 

Once IIoT knowledge has become part of the Hadoop based Data Lake, all the rich analytics, machine learning and deep learning frameworks, tools and libraries now become available to Data Scientists and Analysts.   They can easily produce insights, dashboards, reports and real-time analytics with IIoT data joined with existing data in the lake including social media data, EDW data, log data.   All your data can be queried with familiar SQL through a variety of interfaces such as Apache Phoenix on HBase, Apache Hive LLAP and Apache Spark SQL.   Using your existing BI tools or the open sourced Apache Zeppelin, you can produce and share live reports.   You can run TensorFlow in containers on YARN for deep learning insights on your images, videos and text data; while running YARN clustered Spark ML pipelines fed by Kafka and NiFi to run streaming machine learning algorithms on trained models.

A range of predictive applications are suitable for this tier. The models themselves should seek to answer business questions around things like -Asset failure, the key performance indicators in a manufacturing process and how they’re trending, insurance policy pricing etc. 

Once the device data has been ingested into a modern data lake, key functions that need to be performed include data aggregation, transformation, enriching, filtering, sorting etc.

As one can see, this can get very complex very quick – both from a data storage and processing standpoint. A Cloud based infrastructure with its ability to provide highly scalable compute, network and storage resources is a natural fit to handle bursty IIoT applications. However, IIoT applications add their own diverse requirements of computing infrastructure, namely the ability to accommodate hundreds of kinds of devices and network gateways – which means that IT must be prepared to support a large diversity of operating systems and storage types

The tier is also responsible for the integration of the IIoT environment into the business processes of an enterprise. The IIoT solution ties into existing line-of-business applications and standard software solutions through adapters or Enterprise Application Integration (EAI) and business-to-business (B2B) gateway capabilities. End users in business-to-business or business-to-consumer scenarios will interact with the IIOT solution and the special- purpose IIoT devices through this layer. They may use the IIoT solution or line-of-business system UIs, including apps on personal mobile devices, such as smartphones and tablets.

Security Implementation

The topic of Security is perhaps the most important cross cutting concern across all layers of the IIoT architecture stack. Needless to say, each of the layers must support the strongest data encryption, authentication and authentication capabilities for devices, users and partner applications. Accordingly, capabilities must be provided to ingest and store security feeds, IDS logs for advanced behavioral analytics, server logs, device telemetry. These feeds must be constantly analyzed across three domains – the Device domain, the Business domain and the IT domain. The below blogpost delves into some of these themes and is a good read to get a deeper handle on this issue from a SOC (security operations center) standpoint.

An Enterprise Wide Framework for Digital Cybersecurity..(4/4)

Conclusion

It is evident from the above that IIoT will enormous opportunity for businesses globally. It will also create layers of complexity and opportunity for Enterprise IT. The creation of smart digital services on the data served up will further depend on the vertical industries. Whatever be the kind of business model – whether tracking behavior, location sensitive pricing, business process automation etc – the end goal of IT architecture should be to create enterprise business applications that are ultimately data native and analytics driven.

DZone-GuideToBigData-Apr17

What Blockchain can do for The Internet Of Things..

Blockchain and IoT are a marriage made in heaven. Blockchain can enable & augment a variety of application scenarios and usecases for the IoT. No longer are such possibilities too futuristic – as we will discuss in this post.

IoT meets Blockchain..

Blockchain and Internet Of Things (IoT) are easily the two biggest buzzwords in technology at the moment. The IoT encompasses the world of sensors,moving objects like vehicles & really any device that has embedded electronics  to communicate with the outside world – typically over an IP protocol.

Combine that with Blockchain – a distributed ledger architecture (DLT) pattern.Combining the two can facilitate the entire lifecycle of IoT devices & applications and prove to be the glue for business processes to act on these events. Consider the following scenario – a private blockchain for a driverless connected car that will enable secure and realtime interactions from the car starting with car startup, driver authentication, smart contracts to exchange insurance & maintenance service information and realtime location info to track safety.

Blockchain based distributed ledger technology (DLT) fills in five critical gaps in IoT..

  1. In such typical scenarios as the above, a Blockchain based distributed ledger provides the trust, record of ownership, transparency and the overall (decentralized) communication backbone for IoT.
  2. It needs to be noted that over the years specific IoT communities will develop their own private blockchains that can store transactions in a highly secure manner for their specific applications. IoT architectures that relied on centralized servers to collect and store data will be able to write to local ledgers that will synch with other localized ledgers to maintain a single yet secure copy of the truth.
  3. All IoT transactions on the Blockchain will be timestamped thus ensuring that they are available essentially – for posterity.
  4. Next up, the true innovation of Blockchain – digital agreements or Smart Contracts. Smart contracts can then be applied on the data in the blockchain to enforce business conditions on the IoT interactions.
  5. Finally, one of the big knocks against IoT has been the relative lack of security standards. Blockchain due to its background on high end cryptography actually helps with IoT security. A future post will discuss such a reference architecture.

With that background, let us consider low hanging usecases across key IoT applications in verticals.

blockchain_iot

  1. Industrial Manufacturing – The manufacturing industry is moving to an entirely virtual world across its lifecycle, ranging from product development, customer demand monitoring to production to inventory management. As devices & systems become more interactive and intelligent, the blockchain can serve as a plant level, regional level and global supply chain level ledger. This will dramatically cut costs and drive more efficient just in time (JIT) processes enabling better usage of plant capacity and improved operational efficiencies.
  2. Connected and Driverless VehiclesThe Connected Vehicle enables the car or truck to behave as a giant Smart App. With the passing of every year, vehicles have more automatic features builtin – ranging from navigation, roadside assistance etc. Blockchain will enable these devices to be tracked on the digital mesh thus enabling easy inter vehicle communication as well as automatic tracking of fleet insurance policies, vehicle registration renewals etc
  3. Transportation – IoT + Blockchain = Connected Transportation. A range of scenarios can be imagined around a connected mesh of vehicles that exchange everything from traffic information to avoiding gridlocks & bottlenecks. Extending this to global trade, this mesh can incorporate shipping, air freight as well as ground transportation to track shipments.
  4. Public Infrastructure & Smart CitiesSmart devices are already being used to track the health of bridges, roads, power grids etc. Blockchains can be used to interconnect these to share efficiencies and to conduct maintenance, forecast usage trends for power usage, pollution etc. Another key area of usage would be to help remote areas such as forests to monitor natural incidents and to prevent catastrophic occurrences like large scale forest fires or farm infestations by blight etc.
  5. Financial services and insurance – Banks could use Blockchain backbone to track IoT enabled devices like ATM machines, remote tellers to conduct maintenance. Insurance companies which have already started deploying drones to verify property claims in remote areas can use the Blockchain to validate and verify claims.
  6. Home and Commercial Realestate management – Using sensors deployed on both homes and commercial buildings helps with automated home and office monitoring. The usecases will diverge across both areas but many can be built on having a foundational distributed ledger capability.
  7. Smart Contracts –  Blockchain based cryptocurrencies like Bitcoin enable two business capabilities at a high level – a) transfer of cryptocurrency and b) business rules that specify when the payout has to happen – typically once conditions that have been met – which satisfy fulfillment of contractual terms.
    These rules are termed “Smart Contracts’. Smart contracts are applicable across all of these business areas and can be used to keep track of business rules and take actions based on thresholds that have been met or have been breached. E.g A driverless vehicle that has failed an inspection can be grounded, non payment of home owners insurance can trigger an alert to the homeowners housing society etc.
  8. Retail –  Retailers are already using IoT devices and endpoints to help across the business lifecycle – ranging from the shop floor, to tracking product delivery to store, to understand their customer traffic patterns, wearables etc. The vision of the Connected Store with IoT enabled shelves, an ability for customers to perform more actions using smartphone apps to reducing checkout times with self checkout etc are all taking place. The Blockchain can augment all of these usecases by providing the critical link between retailer and consumer in a way that it automates away the middle man- be it a credit card issuer, or a  central server. For instance consumers can store their product preferences, sizes in a Blockchain and the retailer can access these in a seamless and secure manner.

There still exist large technology shortcomings..

Finally, it needs to be mentioned that there still exist critical gaps in Blockchain technology – whether one considers the public Blockchain on which Bitcoin is built or technologies like Etherium – in terms of interoperability, security standards, throughput and mature developer tooling. These will need to be worked on over the next few quarters before we see production grade IoT deployments on Blockchains.

Conclusion..

The potential ability of Blockchain to ultimately enable secure, global & decentralized communication across billions of IoT endpoints is very promising. As Blockchain matures and makes it way into the enterprise, it likely to impact many aspects of business operations and strategies in the coming years.

How the Blockchain will lead disruption across industry..(5/5)

The first four blogposts in this series have covered the business foundations and the technology behind both Bitcoin and it’s core technology platform – Blockchain.This final post will bring it all together and will cover business models that will either be created or upended as part of this potentially immense disruption.

Though Bitcoin the currency has had the lion’s share of media and public interest over the past few years, it is no longer a point of contention to say that Blockchain technology platform is the more valuable invention & one that will drive industry wide change. In the last post, we discussed how Blockchain can be leveraged as a design pattern to create digitally native & truly distributed applications.

Will there be one Blockchain to conquer them all? Not at all, the kind or flavor of blockchain will be dictated by the business need. One could have a public blockchain used by Government or a private blockchain/ sidechain used by a globally distributed organization/holding company or even vertical blockchains – run by a given industry like Banking & financial services, Retail, Manufacturing. IoT, Healthcare or Govt .

To be sure, the current state of Blockchain still has a high degree of technology challenges (throughput, latency, overall size of the blockchain, lack of developer friendly APIs, multiple forks etc) that must be overcome before it becomes mainstream but all open source technologies eventually get there especially from a throughput, latency & performance standpoint.

However, the obvious commercial success players of Red Hat, Hortonworks & Pivotal Labs across sectors as diverse as operating systems, middleware, workflow engines, the full data stack & cloud computing only bears this out. Mature, stable and feature rich open source technologies outlast commercial & proprietary platforms from an adoption standpoint. Blockchain and it’s growing ecosystem are no different.

At a minimum any application developed using the blockchain as a foundation will have the below inherent technology abilities –

  • Support massive (read global) scale  of deployment – well above and beyond what we term “Big Data”. Any app built on a Blockchain API will inherently be deployable & distributable at massive massive scale
  • Highest degree of trust and security due to cryptography thus eliminating the need for trusted third parties, intermediaries, hierarchies and unnecessary regulatory regimes
  • The ability to provide transparent access to individuals or applications with the right permissions by using a simple (digital) private key
  • Support complete programmability of any artifact (e.g a contract, a deed, a financial instrument, a medication record etc) stored in it
  • Provide complete audit-ability in terms of ownership and access tracking

BC_Evolution

Illustration: Blockchain as the foundation for gradual disruption across Verticals

Given the capabilities shown in the above diagram, the true promise of Blockchain would be to be serve as the seamless economic layer that would be the underlying infrastructure for a whole layer of usecases across industry sectors.

When development in 2.0 streams matures with projects like Ethereum (and even Ripple) enabling developers to create manner kind of application or script (e.g digital currency, fileserving, messaging etc) in away that it can natively  run on a blockchain-like infrastructure. Ethereum , for one, deserves a huge mention as they’re essentially providing digital currency capabilities with smart contracts baked in from the get go.

Established players like Banks are not sitting still. They’ve begun forming consortiums like R3 and also have begun investing into players like Chain who are applying an enterprise model to Blockchain.

To enumerate some of these innovation led blockchain usecases –

FSI, Retail & Insurance – The financial industry is built on a system intermediaries across the spectrum, ranging from Clearing Houses to Payment Processor Networks. Intermediaries  facilitate trust and also to make overall communication secure. However, eliminating or reducing the usage of third parties can save hundreds of billions of dollars every year while facilitating real time interactions e.g. settlements – which is virtually unheard of.

  • Institutional (macro) digital payments,
  • Consumer (micro) & retail payments
  • Eliminate or simplify Capital Markets across the spectrum. E.g financial trading brokering & near real time settlement & clearing in all manners of instruments – equities, fixed income, derivatives etc;  (which the DTCC paper is referring to)
  • Vastly simplify the signing & authentication of digital documents by turning them into smart contracts s e.g. Financial contracts
  • Vastly simplify Chain of Custody & Asset tracking
  • Trade Clearing & Settlement (eliminate the middleman as well as settlement time with current system of paper certificates)
  • Automatic applications and claims management
  • Shrinking the insurance industry value chain across P&C,Life & Health
  • Eliminate the escrow system in mortgage banking. Blockchain will change the face of Mortgage banking which is a significant percentage of any Financial supermarket. Eg JP Morgan, BofA etc
  • Drive the digital transformation by automating and digitizing entire processes e.g home buying
  • Radically simplify Compliance challenges – KYC(Know Your Customer), BSA (Bank Secrecy Act), Fraud & Risk Data Consortiums layered on Blockchain
  • Blockchain oriented applications & data architectures as a way of enhancing well understood usecases around AML, Risk (esp Credit Risk etc), Digital Banking etc – futuristic.
  • The fraud detection space is a no brainer for blockchain.

HealthcareA global patient record system as well as population health management systems can save millions of lives every year while removing a lot of systemic inefficiency that plagues healthcare and life sciences across a spectrum of usecases.

  • Creating Blockchain based EMR’s that span local and regional jurisdictions
  • Anonymize EMR data and feed it into druge discovery and clinical trial processes
  • Speeded up drug discovery & drug trials right from identification of populations to enabling individuals to volunteer for trails
  • Population Health Management within national and local jurisdictions
  • Making the healthcare delivery and pharma sectors much more efficient. e.g Improving availability of medicine supply chains
  • Assessing global diseases outbreak in seconds across the planet..creating a realtime understanding of illness across the globe
  • Improving genome sequencing
  • As a whole, improve preventive medicine

TelecomThe Telecom industry suffers from extreme customer dissatisfaction due to the amount of time taken to provision new services as well as with delays in providing instantly available and highly responsive applications.

  • Enabling easy discovery and signup of telco services. e.g cellphone contracts
  • Simplified service provisioning
  • Easier monetization of digital services and applications
  • Simplify all manner of financial billing
  • Reduce delays in customer on boarding, preference detection and network provisioning

Travel & EntertainmentThe Travel industry is perceived to have a very low degree of transparency with regard to prices and availability of bookings. It is also built upon a system of reward points that keep customers loyal. Both these areas are highly inefficient and are areas of lock-in for customers. The entertainment industry on the other hand relies on intermediaries like iTunes and Cable companies leading to heartburn among artists. The Blockchain model provides freedom of choice to both artistes and consumers.

  • Move rewards to an online system of Bitcoin-like rewards and tracking
  • With strong digital IP tracking – completely replace the current recording industry by enabling groups of artists to register, self publish & collect payment on their creations
  • Migrating Physical asset keys to the BC – Hotel keys, Auto rental records etc
  • Signing & authentication of digital documents by using smart tokens e.g. Contracts
  • Enable embedding of smart contracts in actual physical objects. E.g Lock access to a credit card if the user defaults on the account etc
  • Instant and on demand access to entertainment
  • Easy flight bookings
  • Connected airports and hotels

GovernmentSimplified record keeping for government can save tens of billions of dollars in corruption in the developing world and can lead to better services for the global population.

  • Serve as the authoritative registry for all kinds of record keeping & attestations – births, deaths, marriages, property deeds etc; hugely eliminate corruption in developing countries
  • All manner of notary services
  • Enable banking services for underbanked populations
  • Eliminate some of the current pain in the current electoral process with e-Voting
  • Improve the effectiveness of charity..everything in the blockchain is auditable and it is tamper proof

IoT 

Blockchain and IoT are a marriage made in heaven. Facilitate the entire lifecycle of IoT applications. E.g A private blockchain for connected car that will enable secure and realtime interactions ranging from device registration, user authentication, smart contracts to exchange services among one another and location info to track safety

Manufacturing – The manufacturing industry is moving to a virtual world across the lifecycle, ranging from product development to production as systems become more interactive and intelligent. The blockchain can serve as a plant level, regional level and global supply chain level ledger that will dramatically cut costs and drive more efficient just in time (JIT) processes.

  • Improve operational efficiency by improving visibility into core processes
  • Improving value chains themselves by applying advanced analytics onto data inside manufacturing specific blockchains
  • Match supply and demand across organizations
  • Enable better usage of plant capacity
  • Drive liquid marketplaces

The list of use cases goes on.

The Blockchain opportunity for Technology vendors

Where it gets very interesting is in marrying all these capabilities to a Big Data stack. At about 30 GB[1], the current Blockchain is too small for a massively scalable HDFS (Hadoop Distributed File System) like deployment but it is very conceivable that It could grow based on business and technology requirements for increased capabilities. Case in point is the famous debate on increasing the block size as away to dramatically scale the throughput  to match a commercial payment network like VISA’s.

It is very conceivable that in a few years, some of the blockchain variants could rise to Petabyte scale thus spurring interest from players in the Big Data, Application Development and Cloud Computing space .

The other two massive areas of opportunity in my view are creating distributed apps (DApps) that are blockchain & cloud native but written using Hadoop semantics (Storm, Spark etc);

The other area is of course Data Science & Predictive Analytics across all of the above areas.

IT and Management strategy in the face of Blockchain Disruption –

The creation of smart services on a Blockchain-like infrastructure will further depend on the vertical industries that these products serve as well as requirements for the platforms that host them. There is an increased need to watch innovation very closely as it can rapidly put established organizations out of business very soon. The list of such technologies is long… Cloud Computing, Big Data, Mobility, IoT, 3D printing etc.

For a recent list, please see my previous post on Gartner’s top tech trends for 2016 – http://www.vamsitalkstech.com/?p=1244

Blockchain as a platform for innovation is more akin to Big Data than Cloud. How so? In that it essentially changes not just infrastructure (like with Cloud) but also the very nature of the business services themselves – as with Big Data.

My recommendation for industry practitioners would be to follow the below rough process ,which is common to every two speed IT process focused on reaping business benefits from strategic technologies. The definition of ‘strategic’ as in an emerging technology trend that will impact Iong term business thus influencing plans & budgets.

  1. Examine existing business usecases or pain points that could benefit from a decentralized model. These could services being offered by startups or other players unencumbered by legacy investments
  2. Develop architectural and business expertise in this space by developing small prototypes
  3. Begin targeted proof of concepts that are incremental in nature and are intended to offer capabilities to early adopter customers
  4. Partner with commercial vendors in this space; also keep regulators informed of your intentions and get their buy in
  5. Put plans in place to gradually operationalize and mass market these capabilities

As blockchain projects leave the labs and move slowly into proof of concepts and ultimately into production systems, it behooves industry players to look into the technology to a) offer disruptive services in their vertical b) augment existing applications.

Blockchain has the potential to change any industry & the way it does business. Enterprise CXO’s need to look to apply it as a tool in any business venture where data, transparency of business dealings & security are key.

References

[1] “The Blockchain Economy” – Melanie Swann – O’Reilly Press
[2] https://bitcoin.org/en/developer-reference
[3]”Mastering Bitcoin” – Andreas Antonopoulus – O’Reilly Press
[4] https://en.wikipedia.org/wiki/Block_chain_(database)

The Architecture of Blockchain..(4/5)

The easiest, as well as the most comprehensive way, to define the Blockchain (BC) would be to to call it a “a highly secure, resilient, algorithmic & accurate globally distributed ledger (or global database or the biggest filesystem or the largest spreadsheet) that provides a foundational infrastructure pattern to build multiple types of applications that help companies (across every vertical), individuals and society discover new business models, transact, trade & exchange information & assets. “

Why is the Blockchain such a terrific innovation? 

The creation of the Blockchain by the eponymous Satoshi Nakamoto is nothing short of a work of disruptive genius. While Satoshi’s goal was to create a currency (bitcoin) that did not need a central authority to create, validate & maintain, the architecture as laid out provides the chassis for running truly distributed applications that eliminates the need for 3rd party verification – thus establishing a true peer to peer economy.

Let us consider Banking as we have known and understood it for almost 400 years.

Banking is a system of intermediaries across the spectrum – ranging from payment networks (e.g Mastercard, Visa etc) to Clearinghouses in Capital Markets to Banks etc . And the reason these intermediaries exist is to establish trust between two parties who do not know each other. Thus currency or any other financial instrument created without the need for a central authority needs to be protected from fraud & abuse. In Banking, large institutions stand behind and protect payment systems.

The Blockchain serves that purpose in the realm of the Bitcoin and stands behind every bitcoin ever created by maintaining the proof of ownership.The innovation is it’s openness yet it’s security which ensures that the currency is tamper proof.

Prior to Blockchain, two important shortcomings had hindered the development of a truly secure digital currency.

These are  – 

  1. Currency Counterfeiting  – The problem of double spending that allowed the copy and reuse (effectively counterfeiting) of digital currency again and again (much like an mp3 file) after issue. What is different with bitcoin is that once a bitcoin has been purchased or transferred anywhere in the world – ownership is established and recorded authoritatively via the Blockchain . The Blockchain operates at such a massive scale which makes it virtually impossible (and cost prohibitive) to hack or otherwise break into bitcoin. Thus there is no need for a central 3rd party to a issue, authenticate and validate ownership of the currency.
  2. Who is the definitive legal owner of a given unit of Currency ? – For a peer to peer currency to truly work, each member of the millions of global nodes must be in agreement about the ownership of every unit of the currency & the total number of coins any member owns at any given time. This problem of global consensus (also called the Byzantine Generals problem) in a distributed network is solved by the Blockchain. The Blockchain leverages peer to peer communication to achieve consensus on transaction blocks every 10 minutes (avg) while processing newer blocks constantly as business transactions occur using Bitcoin (or any of the other AltCoins).  Specialized nodes called Miners perform the issue of new units of currency while simultaneously performing large & complex calculations. Doing these ensures that the Miner node(s) can mine a block of transactions during the same time. The benefit to the Miners is that they can keep the transaction fees associated with that block as a reward. It is almost impossible for this system to be broken into as it is in the interest of all the miners to form a distributed consensus based on valid block information. This capability is termed as the proof of work

Blockchain Architecture Considerations:

When one examines the architecture of Blockchain, a few important design aspects that need to be discussed are –

  1. The Blockchain platform itself
  2. The role of Nodes in constituting the overall blockchain & the Node discovery process
  3. Transactions that make up the blocks running in the Nodes
  4. Security implementation that generates the Blocks
  5. The process of adding newer blocks to the Chain

BC_WireTransfer

               Illustration 1 –  Wire Transfer of Bitcoin using the Blockchain shared ledger

Lets examine each of these in the context of the current public blockchain in terms of an end to end transaction shown below. It is my expectation that other kinds of blockchain variants will leverage the current technology as a foundational design pattern while layering in their own industry or consortium specific requirements.

Blockchain Platform:

The Blockchain itself is an application that runs on a network of distributed servers. The core application is a transaction database modeled as a secure ledger,  that is shared by all nodes (servers) that run the full stack install. It is thus a 100% decentralized transaction system that acts as a highly transparent ledger. Any node running the Blockchain software can run the entire Blockchain locally. While the Blockchain client uses Google’s LevelDB database to store metadata internally – the Blockchain data can be stored in a flat file or in a relational DB depending on user preferences.

The below architecture diagram broadly captures the 3 main layers of the Blockchain along with their roles.

BCArch

                        Illustration 2 – Blockchain Architectural Layers (Click to enlarge)

Once installed on a server, the full Blockchain client syncs up with other nodes in the network. From then onwards the particular server maintains all and any transaction records conducted using bitcoins (or any other application running on the Blockchain). The integrity and chronological order of the transactions (& the addresses owning the currency) are enforced by strong cryptographic rules. As stated above, the Blockchain software operates at such a massive scale which makes it virtually impossible (and cost prohibitive) to hack or otherwise break into bitcoin or any other application running on it. Thus there is no need for a central 3rd party to a issue, authenticate and validate ownership of the currency.

The nodes in the overall network use the peer-to-peer IP network to process and verify transactions. When several nodes all have the same blocks in their individual databases, they are considered to be in consensus.

Nodes in the Blockchain:

Blockchain is a peer to peer (P2P) network working on the IP protocol on the internet. A P2P network is essentially a flat topology with no centralized node, hierarchy, or special server node. All nodes equally provide & can consume services while collaborting via a consensus service.

As discussed above, the nodes in the blockchain play the role of a Central Bank or a  trusted third party.  Every node maintains a fully replicated copy of a database that contains the payment history of every bitcoin ever created along with ownership information. As transactions happen using the currency, a consensus mechanism essentially dictates how nodes agree on blockchain updates.

The graphic below shows the current number of nodes active in the bitcoin network and their locations on the globe.

BC_Nodes_         Illustration 3 – Blockchain Nodes – Jan 25,2016 [4]

Though the blockchain enforces an equal structure among all the nodes that are part of the network, nodes can play different roles based on their flavor or business intention.

Thus,  there are 4 basic node types as depicted in the below illustration.It is to be noted that all of the node types discover and maintain connections with peers & also validate blocks.

Full nodes maintain a complete copy of the blockchain database and can verify any transaction without the need for an external lookup. On the other end of the spectrum, nodes that only store a subset of the blockchain database verify transactions using a method called Simplified Payment Verification (SPV).

Nodes of type Miners perform the core process by which transactions are confirmed & processed and eventually included in the blockchain. To be confirmed as valid, transactions are first packed in a data structure & format called a Block that has to satisfy strong cryptographic rules that are verified by the blockchain network. Cryptography also prevents already committed blocks from being further modified as doing so would invalidate all following blocks –  which protects system integrity. Mining also creates an implicit meritocracy in that systems which can process transactions faster and more efficiently get credited for them. For applications like bitcoin, rollbacks currently are almost impossible in the vanilla blockchain architecture – which is a current limitation.

The miner node concept is unique to blockchain as it confers it a high level of security because short of controlling more than half the miner nodes affectionately termed the “51% attack ” (which is a computationally impossible feat to perform at scale), the network cannot be compromised or taken over.

Network Protocol Stack: Once nodes are booted up, they perform a peer discovery to contact any other valid node using a given port over TCP. The Blockchain stack is depicted below and is layered on the OSI stack. The Blockchain Message Exchange specifies the handshake logic between nodes as well as the serialization format for messages exchanged over the wire.

The Blockchain Overlay Network provides higher level semantics that allow multiple types of blockchains (public, vertical specific & private blockchains) to co-exist as well as provides management abstraction for the same. Developers will essentially use this layer to extend vanilla blockchain to support other kinds of applications which can leverage the existing bitcoin blockchain to validate their transactions. E.g. Other kinds of virtual currency, sidechains etc.

BC_Network
Illustration 4- Blockchain Network Architecture  (Click to enlarge)

Transactions and Blocks: 

Applications, the first among them being Bitcoin use blockchain to timestamp transactions. The blockchain implementation consists of two kinds of records: transactions and blocks.

Salient features of transactions –

  1. Transactions can be created on the behalf of any client using a Mobile Wallet or any other client application
  2. Transactions contain the actual business data to be stored in the blockchain
  3. Blocks record the sequence of transactions in the blockchain. Transactions are journaled into the blockchain based on specific sequences
  4. Miner nodes create blocks as discussed in the above section

BC_Transaction

                 Illustration 5- Transactions are converted into Blocks  (Click to enlarge)

The above illustration depicts how currency transactions are converted into blocks. As can be seen,once a transaction (typically a kiosk purchase or wire transfer or credit card payment etc) is submitted into the system, a transaction (say Transaction A) is generated that is pushed into the blockchain node network. A miner node(s) intercepts this to do some sanity checking, once done and it is determined that this is a legit transaction, it is fed into a cryptographic hash function as depicted to generate a unique string of digits. It is then also be combined with other transactions as shown. The generated hash is then stored with other metadata into the header of a data structure called a block. The header is key as it becomes the basis for running the hash function again to create a child block.

The hash function is used as the math puzzle that the miners race to solve by looking over virtually trillions of possibilities. The miner that solves it first submits it for a check by other nodes and once confirmed, the block is stored into the blockchain for posterity. The miner node is then credited a very small percentage of the transaction as fees.

While that ends our view into the main architectural underpinnings of Blockchain, the next & final post in this series will take a broad look at the innovation that will likely be unleashed by the  Blockchain across the industry. We will also discuss what steps CXO’s & IT managers need to take to avoid being disrupted by startups in this area. 

References: 

[1] https://blockchain.info/api
[2] https://bitcoin.org/en/developer-reference
[3]”Mastering Bitcoin” – Andreas Antonopoulus – O’Reilly Press
[4]https://bitnodes.21.co
[5] https://en.wikipedia.org/wiki/Bitcoin_network
[6] https://en.wikipedia.org/wiki/Block_chain_(database)

The immense potential of the Blockchain..(3/5)

“We should think about the blockchain as another class of thing like the Internet — a comprehensive information technology with tiered technical levels and multiple classes of applications for any form of asset registry, inventory, and exchange, including every area of finance, economics, and money; hard assets (physical property, homes, cars); and intangible assets (votes, ideas, reputation, intention, health data, information, etc.)…”.  Melanie Swan (2014)

Blockchain-blue

To recap, the fundamental concepts of the Bitcoin again –

  1. The Bitcoin is a digital currency with no physical representation and no central issuing or controlling monetary authority
  2. The ownership of bitcoins is tracked and validated in a global public ledge called the Blockchain. The blockchain tracks every transaction ever made using the bitcoin with new pages being constantly added
  3. Strong cryptographic algorithms are used to authorize transactions. Nodes called ‘Miners’ perform this task.

Blockchain is the #1 trend in financial services from an industry perspective when you consider the below data points-

  1. About a billion dollars of venture capital investment has flown into a range of startups with half of that investment coming in the last 12 months[1]
  2. Forty two of the worlds largest Banks have formed a consortium – “R3” dedicated to researching Blockchain technology [2]
  3. The Linux foundation partnering with technology vendors led by IBM, Swift, Digital Assets & Microsoft are either pioneering open source projects (OpenLedger)[3] as well as offering “Blockchain as a Service” type clouds.
  4. The overarching goal of all of the Open Ledger Project is to make the Blockchain more enterprise grade and feature rich for businesses to adopt. Blockchain development is following that of the Linux kernel which undergirds a range of systems from massive public clouds to enterprise datacenter servers to the Android mobile OS.

What is Blockchain?

The easiest way to define the Blockchain (BC) would be to to call it a “a highly secure, resilient, algorithmic & accurate globally distributed ledger (or global database or the biggest filesystem or the largest spreadsheet) that provides an infrastructure pattern to build multiple types of applications that help companies (across every vertical), individuals and society discover new business models, transact, trade & exchange information & assets. 

The term Blockchain is derived from a design pattern that describes a chain of data blocks that map to individual transactions. Each transaction that is conducted in the real world (e.g a Bitcoin wire transfer) results in the creation of new blocks in the chain. The new blocks so created are done so by calculating a cryptographic hash function of its previous block thus constructing a chain of blocks – hence the name.

As we can see from the above, the Blockchain lies at the heart of the Bitcoin implementation & is the most influential part of the BTC ecosystem. Blockchain is thus both a technology platform and a design pattern to building global & vertical scale industry applications that makes all of the above possible. It can alternatively be described as a database or a global ledger or even as a distributed filesystem.  All of these properties that makes it possible to be used as a platform for digital currency also enable it to indelibly record any kind of transaction – be it a digital currency ,or, a medical record, or, supply chain data, or, a document etc into it.

Blockchain

                                          Illustration: Key Characteristics of Blockchain

The most important characteristics to know about the Blockchain –

  1. Blockchain is the first truly global platform. Anyone with a simple mobile device or a PC can access it using a simple Wallet application – without the need for a middleman. Based on this type of universal access, the Blockchain as a design pattern and business foundation will enable the creation of a range of business uses. The only limit is the ingenuity of thinking.
  2. Virtual currencies are but one of the kinds of highly distributed applications that can be built on blockchain. Blockchain will likely eventually become the embedded economic layer of the Internet permitting not just currency or payment related activities but a whole range of others. Bitcoin & online payments are some of the first applications built on Blockchain. However the killer app is years away from being invented.
  3. Pathbreaking Applications built for Blockchain will upend business models in almost every industry vertical. It will help startups create new business models. We will examine some of these in the final blogpost in this series.
  4. Blockchain is highly secure & immutable thanks to the ingenuity of the architecture as conceived by Satoshi Nakomoto. We will examine the security implementation in the next post but the Miner nodes not only derive hashes of the transaction blocks but also successively combine them into data structures which preclude deriving the data from the hash function.
  5. Blockchain is highly transparent and auditable as transaction blocks are openly available for introspection are published as evidence that demonstrate the integrity of the blocks.This overall paradigm makes it very mathematically intensive and almost computational impossible to create fraudulent new blocks of transactions, as other transactions are continuously being validated.
  6. Blockchain is also highly reliable in that every node running a full configuration maintains a copy of every transaction executed in the system with only an identifier needed to track ownership – a digital address. Anyone can find out the set of transactions and who owns what using a simple web browser application called an explorer.
  7. Due to its flat, peer to peer nature – Blockchain conclusively eliminates the need for any middleman – thus disinter-mediating existing structures in many industry verticals. E.g Financial Services, Supply Chain Management, Real Estate, Healthcare etc. Thus next generation applications and platforms built over the next few decades will need to consider the blockchain.
  8. Blockchain is also digital and introduces the important notion of programmable instruments – be it currency or contracts. An important illustration of the possibilities of blockchain are what are termed – “Smart contracts”. These are rules embedded with a contract that can automatically trigger based on certain conditions being mer. E.g. a credit pre-qualification or a lawyer’s approval etc.
  9. The other point that is not stressed enough is that in addition to the highest levels of security, the blockchain also enables a high velocity of transactions. As an example, if two consumers are connected to the blockchain – one a home buyer & the other a seller, they can find each other, verify the authenticity of the title & any liens on the property, credit pre-qualification etc and finally conduct the transaction in realtime without the need for a middleman (a Bank or a Clearinghouse or a Real estate broker).
  10. Blockchain technology thus has massive potential to be applied in any industry situation where data security & identity theft are major challenges. This includes a plethora of usecases across financial services, retail, IoT, manufacturing, healthcare & real estate.
  11. A point of confusion for a lot of corporations around this technology is the perceived need or dependence around a publicly accessible database – much like a public cloud. However, the Blockchain is an extensible technology paradigm more than an actual platform. To further illustrate this point, there are potentially three types of blockchain implementations possible – a Public Blockchain, a Vertical Industry specific blockchain and a Private Organization blockchain. These could be run inside private clouds with enhanced & business specific security and authentication procedures as applicable thus serving a community of interest.

These key characteristics of blockchain technology open the door to dis-intermediating third parties from myriad business transactions, lowering transaction costs down to a fraction, – ultimately increasing the potential for innovation in every major industry.

In Conclusion – 

To recap, Blockchain is essentially a system & architectural design pattern for recording transactions with the highest degree of security, transparency and anonymity. Bitcoin is but the first application of this ground breaking platform.

Though Bitcoin is the first and the most famous killer app for Blockchain, it is a vastly different way of thinking about existing business verticals and usecases – in terms of efficiency,  reliability, speed & security.

Having said all of this, it bears note that Blockchain is still in its nascent stages of development and has a few years to pass before real world applications can be built on it.

Thus, it is only natural that several questions crop up in the minds of Industry architects, managers and executives.

  1. What are the technology underpinnings of the Blockchain?
  2. What are the major usecases across verticals like Financial Services, Insurance, Healthcare, Manufacturing etc
  3. How can horizontal areas like IoT leverage it?
  4. What are the ramifications for Applications, Data Architectures & Infrastructure?
  5. What should our strategy be in the face of this disruption?

The next post will speak to the technology underpinning the Blockchain. The final post in this series will answer the remaining questions in terms of usecases and business strategy.

References:

[1]  www.coindesk.com/bitcoin-venture-capital/, and Deloitte analysis.

[2] https://en.wikipedia.org/wiki/R3_(company)

[3] http://www.linuxfoundation.org/news-media/announcements/2015/12/linux-foundation-unites-industry-leaders-advance-blockchain

The Architecture of Bitcoin..(2/5)

The first post in this series introduced Bitcoin & the crypto-currency movement that will help drive democratization in the financial industry and society at large in the years to come. This second of five blogposts will discuss Bitcoin (BTC) from a technology architecture perspective. The third and fourth posts will focus on the Blockchain from a business & a technology standpoint. The final post will bring it all together and will cover business models & native application architectures that will be created as part of this immense disruption.

As discussed in the last post, Bitcoin is essentially a novel way of stringing  together existing technology (Cloud Computing, Big Data & Cryptography) to provide a disruptive payment service. The very concept and the software based implementation of Bitcoin were originally proposed in a white paper released (November 2008) by an individual(s) under the eponymous name Satoshi Nakamoto. The paper was titled –  Bitcoin: A Peer-to-Peer Electronic Cash System[1].  In this paper, the author(s) proposes BTC as a digital currency that overcomes two important challenges that bedeviled cryptographic digital currencies prior to the bitcoin.The term Bitcoin (BTC) is a frequently overloaded term referring to a currency, a digital protocol and also an ecosystem of services. This post will focus on the technology underpinnings of BTC.

To reiterate these  – 

  1. The problem of double spending that allowed the copy and reuse (effectively counterfeiting) of digital currency again and again (much like an mp3 file) after issue. What is different with bitcoin is that once a bitcoin has been purchased or transferred anywhere in the world – ownership is established and recorded authoritatively via the Blockchain , a network of distributed servers. The Blockchain operates at such a massive scale which makes it virtually impossible (and cost prohibitive) to hack or otherwise break into bitcoin. Thus there is no need for a central 3rd party to a issue, authenticate and validate ownership of the currency.
  2. For a peer to peer currency to truly work, each member of the millions of global nodes must be in agreement about the ownership of every unit of the currency & the total number of coins any member owns at any given time. This problem of global consensus in a distributed network is solved by the Blockchain. The Blockchain leverages peer to peer communication to achieve consensus on transaction blocks every 10 minutes (avg) while processing newer blocks constantly as business transactions occur using Bitcoin (or any of the other AltCoins).  Specialized nodes called Miners perform the issue of new units of currency while simultaneously performing large & complex calculations. Doing these ensures that the Miner node(s) can mine a block of transactions during the same time. The benefit to the Miners is that they can keep the transaction fees associated with that block as a reward. It is almost impossible for this system to be broken into as it is in the interest of all the miners to form a distributed consensus based on valid block information. This capability is termed as the proof of work

Architecture –

Like most modern computing stacks that provide web scale data & application processing capabilities, the most among them notable being the Hadoop ecosystem and modern Cloud Computing architectures built on OpenStack- Bitcoin is essentially architected as a massive peer-to-peer network.

Bitcoin works on top of the IP protocol that connects the internet together. Peer to Peer (P2P) essentially implies a flat network with no single controller or server node. Each node plays an equal role in providing & consuming services on the behalf of users (and their bitcoin wallets). Early peer to peer networks include BitTorrent & Kazaa etc. The P2P nature of Bitcoin ensures that the supply of the currency is regulated by no one authority and the deflationary property of the Bitcoin system’s money supply is distributed evenly by specialized nodes called miners who not only generate the currency but also help secure the network.

The technical architecture of Bitcoin is depicted in the below illustration and is composed of the following 3 core services –

BCArch

                                               Illustration 1: Bitcoin Architecture

  1. The Blockchain Platform
  2. Bitcoin Protocol & Client Application Layer
  3. Bitcoin Currency Layer

Let’s cover each of these below –

Blockchain Platform
The blockchain is a decentralized database that acts as a highly transparent ledger which maintains all and any transaction records conducted using bitcoins. The integrity and the chronological order of the block chain are enforced with strong cryptography implementation. The Blockchain is comprised of millions of network nodes participating in the Bitcoin network. Each node in the Bitcoin network runs a portion of Blockchain which ensures that every transaction conducted using the Digital Currency is unique across the system.

Bitcoin Miners
Bitcoin mining is the core process by which currency transactions are confirmed & processed and eventually included in the blockchain. To be confirmed as valid, transactions are first packed in a data structure & format called a Block that has to satisfy strong cryptographic rules that are verified by the blockchain network. Cryptography also prevents already committed blocks from being further modified as doing so would invalidate all following blocks –  which protects system integrity. Mining also creates a meritocracy in that systems which can process transactions faster and more efficiently get credited for them. In addition, rollbacks are almost impossible which prevents chargebacks using the bitcoin currency.

Grids of servers called “miners” create bitcoins by constantly computing answers to mathematical problems while simultaneously processing transactions. It is to be noted that any node participating in the bitcoin can be a miner provided it runs the full stack of Bitcoin software. Once it is a designated as a Miner, it can first certify and then record transactions. The Bitcoin Miner thus plays the role of a Central Bank in creating and updating currency. The Bitcoin architecture is designed such that every 10 minutes, a Miner is able to validate transactions conducted in the last 10 minutes and is rewarded with bitcoins. Every 10 minutes on average, a miner(s) is able to validate the transactions of the past 10 minutes and is rewarded with new bitcoins .The Blockchain (BC) is shared by all nodes & is updated by the miners. The BC maintains an ordered and timestamped ledger of all transactions. Cryptography ensures the constant integrity of the Blockchain.

To give readers a technical sense of the current state of the bitcoin mining process, the miner process begins scanning for a value that when hashed twice with a cryptographic hash function like SHA-256 begins with a number of zero bits. Thus, this has the two pronged consequence that the required computational work required goes up  exponentially with the number of leading zero bits required but a hash can always be certified by executing a single round of double SHA-256. Thus, “hard to solve but easy to prove“.

Once the process has produced a valid result, the transaction block cannot be changed without redoing the work. As later records or “blocks” are chained after it, the work to change the block would include redoing the work for each subsequent block.

Bitcoin mining has become a highly resource intensive process with clusters using high end chipsets and CPUs & GPS’s and even FGPA/ASIC technology found in high end architectures.

 Why is Bitcoin so highly secure? 

Majority consensus among the nodes in a bitcoin network is represented by the longest chain, which required the greatest amount of effort to produce it.  It is a given that a majority of computing capacity and power is always  within the limits of honest miner nodes. Past blocks cannot be modified by rogue nodes as the computational capacity needed to do so will be exponential in terms of surpassing the legit nodes.

PS – Blockchain’s internal architecture will be covered in more detail in the next two posts.

Bitcoin Protocol & Client Layer

The Bitcoin protocol provides two main functions –

  • APIs that mediate access to the underlying blockchain platform.
  • Algorithms that drive & control Miners across the network.

Bitcoin’s API Layer  exposes programmatic interfaces & APIs to all the various points of interactions with the BTC system mainly the bitcoin wallet,the blockchain network and transaction services provided by Bitcoin.

Screen Shot 2016-01-14 at 11.39.41 AM

                                          Illustration 2: Bitcoin Developer APIs

A list of APIs is called out below and a full list is available at [1].

Bitcoin’s Client Layer consists of end user software applications that can be installed on a full server or a smartphone or simply used over an internet browser. The client facilitates the generation of security features like private keys and also provides Wallet functionality that facilitates the sending of currency transactions using the client’s private key. As discussed above, the block chain is a shared public ledger on which the entire Bitcoin network relies. All confirmed transactions are included in the block chain. Using APIs provided by the Bitcoin client,  wallets can calculate their currency balance and the blockchain network can verify transactions for account limits.

Bitcoin Wallets

A Wallet is a simple file or database that stores (digital) keys (among other things including addresses & digital signatures) that prove ownership of bitcoins. The keys stored in a user’s wallet are generated and managed by the user’s wallet software without reference to the blockchain or access to the Internet. The usage of keys provides three important functions – a. A model that enables decentralized control b. a high degree of security & c.proof of ownership.

clear-overview

                            Illustration 3: Bitcoin Wallet from the Core Client from Bitcoin.org

The Bitcoin client can take different forms depending on the amount of control the user desires over their currency transactions – payments & transfers etc. To sum up, there are three main forms of Bitcoin clients :

Thick (full) client
The full client also called as a “Full node” is a client that provides the user’s Wallet & stores the entire & complete ledger of transactions using the currency that were ever performed..virtually every transaction. It also provides highly useful information information about the state of the blockchain and (to not put too fine a point to it) the **complete set** transactions conducted using the currency.
Lightweight client
A lightweight client stores the client’s wallet but relies on third-party–owned servers for access to the bitcoin transactions and network. The light client does not store a full copy of all transactions and therefore must trust the third-party servers for transaction validation.
Web client
Web clients as accessed as a service over a web browser. They are the easiest to use but do not provide any of the advanced features the other two types of clients provide. Bitcoin is also very mobile friendly and provides clients for smartphones, that can either operate as full clients, lightweight clients, or simply web clients.

Vendors like mSIGNA build on top of the core APIs to provide advanced yet easy-to-use wallet featuring speed and simplicity, enterprise-level scalability, and strong security. Advanced clients like those of mSIGNA support a range of capabilities beyond the vanilla – multisignature transactions, offline storage, multidevice synchronization, and encrypted electronic and paper backups (src: mSIGNA).

Bitcoin Nodes use the peer-to-peer IP network to process and verify payments. Bitcoins are sent from one address to another using transactions and the network thus forms a highly secure payment network. The nodes can take one of four personas as depicted in the picture below.

BC_Nodes

                      Illustration 4: Four Node Personas in a Bitcoin Network[6]

The “Satoshi” reference client that is downloadable from bitcoin.org is a complete implementation of all node personas  depicted above. It includes a complete copy of the bitcoin ledger & filesystem, client wallets, a transaction verification engine. It plays the role of a full network node in the peer-to-peer bitcoin network. A client configured as as a full client joins up the blockchain network and downloads the complete transactions onto itself in a process that could take anywhere from hours to days.Such nodes discover and connect up with their neighbors in the p2p architecture, then receive, validate and then propagate transactions and the relevant blocks. This routing functionality is depicted as the “Routing Node” in the above picture.

Bitcoin Transactions:

Transactions are messages that denote business actions using bitcoins. Naturally, they are are the heart of the bitcoin system.  Client applications create transactions that are then propagated into the network, processed & committed by Miners and then finally added to the indelible record that is the Blockchain ledger. Transactions are essentially data structures that encode the transfer of value between participants in the bitcoin system.There are multiple types of transactions depending on the use case ( a credit card payment, wire transfer & a payroll transaction sent to hundreds of receivers)  but they all have one or more Inputs and one or more Outputs. This is to ensure that bitcoins can be split and combined. Common transactions will have either a single input from a larger previous transaction or multiple inputs combining smaller amounts, and one or two outputs: the first one for the payment of the transaction, and second one returning the change owed to the consumer. The transaction fee to the miners is the difference between the input and output amounts.

Screen Shot 2016-01-13 at 10.28.34 PM

                                              Illustration 5: Blockchain’s Transaction Explorer[4]

Bring it all together

Let us now consider an end to end  BTC Wire Transfer Illustrated using all the concepts discussed above – Wallets, Transactions,Miners & the Blockchain.

The below diagram illustrated how the process works end to end while bringing together all the technical building blocks discussed above & is largely self explanatory-

BC_WireTransfer

                                      Illustration 6: End to End Wire Transfer using BitCoin

References: 

[1] https://blockchain.info/api
[2] https://bitcoin.org/en/developer-reference
[3]https://bitcoin.org/en/bitcoin-core/
[4]https://blockexplorer.com/
[5] https://en.wikipedia.org/wiki/Bitcoin_network
[6] “Mastering Bitcoin” – Andreas Antonopoulus – O’Reilly Press

Bitcoin (BTC) Ushers in the Future of Finance..(1/5)

The decentralized, peer to peer,high secure and purely digital currency finally came of age in 2015. Bitcoin (BTC) & other variants (the AltCoins) got widespread & positive notice by all industry actors ranging from Consumers, Banking Institutions, Retailers & Regulators. Riding on the real pathbreaker – the Blockchain, the crypto-currency movement will help drive democratization in the financial industry and society at large in the years to come. This first of five blogposts will discuss Bitcoin (BTC) from a business & technology standpoint. The second post will take a deepdive look at the technology behind BTC. The third and fourth posts will focus on the Blockchain. The final post will bring it all together and will cover business models & native application architectures that will be created as part of this immense disruption.

BC_Logotype

If there is something that touches all of humankind everyday across the planet & plays a central role in our existence – it is money & the monetary system. Money supply is the backbone of the financial system & as is typically considered a safe asset that households and businesses can use to meet their financial obligations (e.g payments) or to hold as short-term investments (e.g.savings & checking accounts).

Let us understand how the typical banking system has been structured from a monetary & historical perspective since the 1600s but is still very relevant even now. Please note that this is a very high level and basic explanation that segues into our examination of Bitcoin.

A Quick Primer On How Money Works – 

The simplest and most commonly accepted form of money is currency – a dollar bill or a euro coin etc.This paper representation can be converted into any commodity of suitable value. E.g  Lunch or a Cup of coffee or even downpayment on a Car etc. 

When the monetary system was first created,the unit of monetary value – typically a coin- had some intrinsic value in and of itself. It was either made of silver or copper or nickel etc & could be used to support the currency in the event of a bank run or economic downturn. The limitation with this way of doing things was that since the supply of silver or copper in the world is fairly limited, the money supply has no way of growing to accommodate an expansionary economy. 

After a few decades, the Gold standard was adopted which dictated that the money supply was backed up by a fixed quantity of gold. Any government adopting the gold standard guaranteed a fixed exchange rate with the currency of another country that uses a similar standard.  The Gold Standard worked well for decades.

However, as economies became more complex and increasingly non farm based, the inflexibility inherent in the gold standard was a big problem. Case in point, it was partly blamed for the Great Depression. After the war, the Bretton Woods conferences of 1944 (https://en.wikipedia.org/wiki/Bretton_Woods_Conference) helped create the International Monetary Fund with the aim of enforcing a set of fixed exchange rates that were linked to the dollar.However, in 1972 the Nixon Administration initiated a series of economic measures (in response to economic headwinds & unfavorable currency exchange rate) that disengaged the US from the Bretton Woods System. Essentially, direct convertibility of the US dollar to Gold was canceled rather unilaterally – which rendered the Bretton Woods agreements inoperative.

In a quick span of two years by 1973-74, the Bretton Woods system was replaced by the era of freely floating fiat currencies – not backed by any commodity but solely by the faith in the Central Bank & Government of the Country in question.

The point to then take note of is that a currency note of today  (made of paper) has no intrinsic value in and of itself – in that it cannot be redeemed for any commodity – if society & economic order collapsed. The value of a dollar bill is typically endowed by an issuing & regulatory authority typically the Central Bank such as the US Federal Reserve. This Central regulatory authority prints the currency, controls its supply based on macroeconomic conditions & recognizes it as legal tender for all transactions – public or private. Thus this kind of money is called fiat currency – in that the value of the currency is set by fiat (a Government mandated order). Fiat Currencies form the bedrock of the modern day fractional banking system.

Why Governments need to control the Currency Supply? 

The most important reason is that the money supply is the most critical lever in the Government being able to counter business cycles and to temper recessions & booms. Generally speaking, there are two broad ways to defend a present day economy that is typically complex & hugely diversified in terms of private sector participation –  monetary policy and fiscal policy. While fiscal policy deals with public tax policy which have a lagging effect and take time due to the political process involved – monetary policy can almost instantaneously produce the desired economic effect.

Thus, Monetary policy is the core function of Central banks and is concerned with setting benchmark interest rates (e.g the federal funds rate in the US) that directly influences the cost of credit throughout the economy. The biggest advantages of fiat money is being able to inflate it’s value (by printing more bills) to counter economic cycles or for other political purposes (e.g Zimbabwe Dollar, Argentina Peso & Weimar era Germany among the notable examples).

Fast forward to the financial crisis of 2008, governments all over the world were forced to inflate the value of their currencies by resorting to massive creation and printing of Dollars,Euros,the British pound etc. This inflationary or rather expansionary view of the economy originates with Keynesian (https://en.wikipedia.org/wiki/John_Maynard_Keynes) thinking and is in direct conflict with the libertarian movement associated with Hayek (https://en.wikipedia.org/wiki/Friedrich_Hayek) – which espouses the Gold standard.

As computer technology advanced over the years, the field of cryptography was originally used to ensure secure private communication between two systems or people who had a trusted business relationship. Its scope has since expanded to encompass  usecases as diverse as integrity for both data in motion & at rest, telemedicine, electronic transactions, digital cash, secure & massive distributed computation etc.

Bitcoin (BTC) has strong origins in the Cypherphunk movement  which itself is rooted in libertarian thinking. The goal of the Cypherphunk’s is to influence societal and economic change through the use of cryptography. [2] 

The Origin of Bitcoin

So what is Bitcoin? Bitcoin is essentially a novel way of stringing  together existing technology  (Cloud Computing, Big Data & Cryptography) to provide a disruptive service. The very concept and the software based implementation of Bitcoin were originally proposed in a white paper released (November 2008) by an individual(s) under the eponymous name Satoshi Nakamoto. The paper was titled –  Bitcoin: A Peer-to-Peer Electronic Cash System[1].  In this paper, the author(s) proposes BTC as a digital currency that overcomes two important challenges that bedeviled cryptographic digital currencies prior to the bitcoin.

These are  – 

  1. The problem of double spending that allowed the copy and reuse (effectively counterfeiting) of digital currency again and again (much like an mp3 file) after issue. What is different with bitcoin is that once a bitcoin has been purchased or transferred anywhere in the world – ownership is established and recorded authoritatively via the Blockchain , a network of distributed servers. The Blockchain operates at such a massive scale which makes it virtually impossible (and cost prohibitive) to hack or otherwise break into bitcoin. Thus there is no need for a central 3rd party to a issue, authenticate and validate ownership of the currency.
  2. For a peer to peer currency to truly work, each member of the millions of global nodes must be in agreement about the ownership of every unit of the currency & the total number of coins any member owns at any given time. This problem of global consensus in a distributed network is solved by the Blockchain. The Blockchain leverages peer to peer communication to achieve consensus on transaction blocks every 10 minutes (avg) while processing newer blocks constantly as business transactions occur using Bitcoin (or any of the other AltCoins).  Specialized nodes called Miners perform the issue of new units of currency while simultaneously performing large & complex calculations. Doing these ensures that the Miner node(s) can mine a block of transactions during the same time. The benefit to the Miners is that they can keep the transaction fees associated with that block as a reward. It is almost impossible for this system to be broken into as it is in the interest of all the miners to form a distributed consensus based on valid block information. This capability is termed as the proof of work

Since 2009, BTC has been widely accepted at a growing list of online merchants and Banks. A comprehensive list is maintained here [2] & includes players ranging from Amazon, HomeDepot, eBay to Expedia etc. Bitcoin is open source in nature and is essentially controlled by no one but by the consensus of the community.

What is the Bitcoin? 

So how does Bitcoin approach some of the limitations of traditional currency as we know it ? The central theme of bitcoin is its peer to peer (P2P) nature (driven by technology) where the total and complete absence of any issuing or central authority enables it to serve as a total digital ecosystem. As noted above, it has strong origins in a technocratic- libertarian way of thinking in that it democratizes the money supply while opening up vast new possibilities and business models.

Make no mistake, the original concepts behind BTC are extremely elegant and beautifully designed.

Please note that Bitcoin is a frequently overloaded term referring to a currency, a digital protocol and also an ecosystem of services. We will cover the currency aspect here, the next few posts will focus on the technology underpinnings and the ecosystem that it enables – along with the blockchain.

Let us enumerate the defining characteristics of the Bitcoin (BTC) –

1. The BTC is a virtual currency in that it is entirely digital – with no physical representation like paper or coins – and is created online using a process called “mining”.  BTC’s trade like any financial instrument and are issued on a fixed basis with an upward limit of 21 million units till 2040.
2.Bitcoins are peer to peer & not created or regulated or backed by any central authority – bank or government regulators (and their whims & fancies). It is technocratic and democratic at once. Technology operating at internet scale creates & transfers control of bitcoins to consumers & merchants as a way of buying goods and services. BTC will exist as long as the internet exists and it’s value is purely governed by supply and demand.
3. Advances in cryptography and distributed computing making it almost impossible to hack the BTC. It is thus much more safer than traditional currency to steal or counterfeit or to use over internet transactions. Further, transactions performed using bitcoin can be done by supplying minimal information which makes it extremely hard for malicious actors to hack into or otherwise modify it. Underlying BTC is the concept of a Blockchain.
The Blockchain is a shared public ledger or a massive & highly reliable database which serves as the foundation for the entire Bitcoin distributed network.Every time a user spends their bitcoins or millibits from their BTC wallet – these transactions are confirmed & recorded into the immutable blockchain with the highest guarantees for integrity & chronological order.
4. The BTC is deflationary in nature & this is by design. The total number of bitcoins that can every be created are limited to 21 million till 2040. This fixed number does not inhibit the creation of new bitcoins as they have an infinite capacity to be broken down into smaller divisions. This means that bitcoin cannot be used as a traditional currency can be – as discussed above – e.g. inflate it’s value to pay off debt.
5..Every transaction made using bitcoin is reliably recorded in an un-erasable public ledger the Blockchain (more on this in the next post). This opens up it’s applicability to a vast array of business capabilities.
5.BTC transactions are processed by independent (yet interconnected grids of servers called ‘miners‘). The advantage conferred by this approach is that a vast number of miners keeps the transaction fees very very low to 0.001 bitcoins. Hence bitcoin removes the omnipresent intermediary in financial services
6.BTC transactions while performed at a retail setting may seem slow taking upto a few mts for verification by the Miner infrastructure. However, its true potential becomes evident with use-cases like wire transfers, cross border payments etc which can take days with conventional banking networks just take a few seconds to minutes using BTC.
7.BTC supports the entire spectrum of the money supply employed in a fractional reserve system..M0,M1,M2 etc. It supports this notion by supplying off the chain transactions (more on that in the next post)
We will now examine how BTC works in a real world setting.
How does a traditional Credit Card Transaction Work

Let us consider the most common form of online transaction –  a credit card payment. Credit Card payment networks are high cost pipelines both from a consumer and a merchant perspective. Merchants complain of constantly increasingly interchange fees and consumers accuse it of predatory lending practices.

The below pictorial explains a highly simplified approval process (not clearing or settlement mind you).

Traditional_jpeg

                                    Illustration 1 – Traditional Credit Card Transaction 
How does a BitCoin Transaction Work
All one needs to transact using Bitcoins is an address, a wallet and a private key. There is no need to register with a central authority or an intermediary as transactions are verified using internet based “miners” All of these are completely software based thus making bitcoin a true digital currency. As can be seen below in Illustration 2 – the same transaction with BTC is vastly simplified,
Bitcoin-jp
                                    Illustration 2 – Credit Card Transaction performed using Bitcoin

Advantages of Bitcoin

  1. BTC promotes complete transparency & neutrality as to the creation,usage and movement of BTC. All transactions and bitcons issues can be tracked in realtime using APIs or mobile clients and 3rd party services
  2. Enables complete freedom from the intermediary structure that have been built up in Banking over the years. Different versions of this exist across the spectrum  ranging from Clearinghouses in Capital Markets, to Custodial Banking to the Payments industry. BTC enables the supplanting or even complete replacement of these structures
  3. Very low transaction fees compared to traditional payment networks and clearinghouses
  4. High speed of transaction settlement and contracts transfer compared to traditional clearinghouses
  5. Highest level of security owing to it’s complete transparency –  a huge selling point in these days of constant cyber theft and security attacks.
  6. BTC eliminates the dual challenge that plagued digital currencies before it. These are the problem of double spending and the problem of distributed consensus
  7. Potential lower regulation due to all of the above 

Disadvantages of Bitcoin

  1. Much lower degree of acceptance across the world as compared to traditional fiat currency
  2. Higher volatility as can be seen from the last few years
  3. Immature currency trading markets that need time to evolve
  4. Will supplant but never completely replace fiat currency 
  5. Scalability needs to be improved to matchup with e.g. Credit Card networks etc in the coming years

Summary

Bitcoin will never quite replace fiat currency and nor should aim to do so. As has been termed – it aims to be the currency of transactions In the online world affording consumers & merchants an enormous amount of independence from existing structures. Bitcoin can be employed across a diverse range of use cases (which we will examine in the last post in this series) ranging from online payments to recording complex financial transactions to Contracts to Clearing & Settlement. BTC (and a full blown ecosystem around it) can be truly disruptive in any business situation involving financial intermediaries.

However it’s true potential is being able to bring the vast percentage of the global population that has no access to the banking system into it. With Bitcoin, all one needs is a a cellphone, a bitcoin account and a Wallet app  to be able to begin participating in the financial system.

Much of the innovation around bitcoin is driven by the Blockchain.  The next post will be a deep technical look into the Bitcoin after which point we will move onto the Blockchain – the true disruptive innovation.

References

[1] https://bitcoin.org/bitcoin.pdf

[2] https://en.wikipedia.org/wiki/Cypherpunk