Stream processing is an extremely powerful data processing paradigm which helps us process massive amounts of data points and records in the form of a continuous stream and allows us to achieve real time processing speeds.

Apache Kafka is an extremely powerful tool when it comes to data streams. It is highly scalable and reliable. [1]

Elasticsearch is a search engine based on the leucine library. It provides a distributed, full text search engine with an HTTP web interface and schema-free JSON documents. [2]

Kibana is an excellent tool for visualising the contents of our elasticsearch database/index. It helps us…

Python is fantastic language for learning objected-oriented programming. An important pillar of objected-oriented programming is using inheritance in your code. Writing two classes which represent a similar real world object but bear not relationship with each other is actually on not much use.

This is the short blog on how to make the best use of classes in python.

Let’s look at the example below. The class vehicles has two variables called manufacturer and color. The methods set_manufacturer, set_color and set_rating is used to update the variables for this class.

We will use the vehicles class as the parent…

As the number of digital data transactions are increasing, it is becoming extremely important to get some insights out of this ever increasing data pile and organizations are looking towards to creating analytics of this data in a real-time manner.

Many analytics engines and systems use batch processing to perform operations on this data. But a drawback of batch processing is the delay which comes with it. Streams are continuous data chunks which flow from one node of operation to another using a channel. Streams help us generate results and predictions on data points at every point of time for…

Rest APIs are beautiful pieces of software that enable applications to communicate with a database or with other software very effectively.

The kind of operations which you can perform are as follows are GET, POST, PUT, DELETE, TRACE, etc. The GET method is generally used for fetching some information from the backend. For example, if an app (web/phone) wants to fetch some information from the database, it will use the GET method. POST is used for creating new information. PUT is used for updating already existing records. DELETE, as the name suggests is used for the deletion of records in…

PyTorch Logo (

I decided to revisit the concepts of deep learning and chose PyTorch as a framework for this task. I started off with the implementation of a basic neural network in PyTorch using the various tools this framework provides such as Dataloader, the nn module and LR scheduler and more.

I have been mostly using Tensorflow and Keras in my deep learning journey so far. The upgrades that Tensorflow 2 brings with itself are great. But I have to say this, I am just loving PyTorch. Being a python programmer, creating Deep Learning training and inference codes haven’t been so cleaner…

At many stages during a developer’s career, he/she has to build systems capable of communicating with another. For example, there may be two python programs and the second program’s operations might depend on the output of the first program. There can be multiple programs in that linear sequence, with every consecutive program depending on the output of the previous. The linear sequence may even branch out to two child programs depending on the output of the previous. An example dependency graph is shown below.

Fig 1: Dependency graph for python programs

As shown in Fig 1, program 2 depends on the output of program 1. Similarly, program…

I have come across many developers who face serious issues when it comes to installing tensorflow on a linux distro such as Ubuntu. There are very few instances when the installation goes smoothly the first time itself. Mostly, the developer has to face a plethora of error messages which get quite tricky to solve. Some of the error messages are shown below:

[...\stream_executor\] Couldn't open CUDA library nvcuda.dll[...\stream_executor\cuda\] Unable to load cuDNN DSO

Similar error messages can be found in this official tensorflow link →

Let’s make things easier and lives simpler:

Enter Docker:

Using Docker containers the life of…

Building Deep Learning networks is a thing and deploying them for production is a completely different paradigm. Developers spend days scratching their heads over deploying the model and getting all dependencies correct.

We will go over a problem statement of building an object detector using the YOLO object detection framework and deploying it as a docker container. …

It is late 2019 and Deep Learning is not a buzzword anymore. It is significantly used in the technology industry to attain feats of wonders which traditional machine learning and logic based techniques would take a longer time to achieve.

The main ingredient in Deep Learning are Neural Networks, which are computation units called neurons, connected in a specific fashion to perform the task of learning and understanding data. When these networks become extremely deep and sophisticated, they are referred to as Deep Neural Networks and thus Deep Learning is performed.

Neural Networks are so called because they are speculated…

Abhishek Bose

Engineering at NAYAN Technologies. On a quest for technology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store