Cotygodniowa dawka linków, czyli archiwum newslettera Dane i Analizy
Orchestrate a Data Science Project in Python With Prefect
Optimize Your Data Science Workflow in a Few Lines of Code Motivation As a data scientist, why should you care about optimizing your data science workflow? Let’s start with an example of a basic data science project. Imagine you were working with an Iris dataset. You started with building functions to process your data. https://medium.com/media/e479ca8deb87434b1859864118ec3bcc/href After defining (…)
Dealing With Dates in Pandas — 6 Common Operations You Should Know
Dealing With Dates in Pandas — 6 Common Operations You Should Know Never confused with dates again, hopefully. Besides texts and numbers, dates are a very common data type in our datasets. When we use Pandas to process dates, it’s definitely not the most straightforward task for most data scientists. When I began to use Pandas, I wasn’t familiar with date-related features, so every time when I (…)
Remove Text from Images using CV2 and Keras-OCR
How to automatically modify images to make them text-free using Python An example of before and after removing text using Cv2 and Keras. Source: image by the author processing an image by morningbirdphoto from Pixabay . Introduction In this article I will discuss how to quickly remove text from images as a pre-processing step for an image classifier or a multi-modal text and image classifier (…)
Spark Streaming with Python
Your guide to getting started with Pyspark Streaming Pipelines Streaming Architecture of Spark. Spark streaming architecture diagram from author Spark Streaming’s primary structure is batch-by-batch discrete-time streaming. The micro-batches are constantly allocated and analyzed, rather than traveling through the stream processing pipelines one item at a time. As a result, data is distributed to (…)
Performance Comparison: CatBoost vs XGBoost and CatBoost vs LightGBM
Boosting algorithms in machine learning — Part 7 Continue reading on Towards Data Science »
Use Julia For Awesome Data Science Projects
Know what Julia has to offer, and how it fares in front of Python Image by author As part of maintaining a versatile toolbox and being up to date with tech progression at our company, we sometimes set out on a journey to try new technologies, which may also include programming languages. Julia (programming language) was long on my radar, since version 0.6 or so, but I never felt it was fully (…)
Introduction to Aggregation Functions in Apache Spark
This article was published as a part of the Data Science Blogathon. Introduction Aggregating is the process of getting some data together and it is considered an important concept in big data analytics. You need to define a key or grouping in aggregation. You can also define an aggregation function that specifies how the transformations […] The post Introduction to Aggregation Functions in Apache (…)
Top R Packages for Visualizing Table Data – Make Stunning Tables in Minutes
Visualizing table data is no small task. It requires both data manipulation and data visualization skills from the technical end. It also requires knowledge about your audience. Ask yourself — For whom are you visualizing the data? Do you need interactivity? Will you include the table in a web application? The list of questions goes […] Article Top R Packages for Visualizing Table Data – Make (…)
Probabilistic Deep Learning for Wind Turbines
How to apply Gaussian Processes on big data Model speed can be a deal breaker on large datasets. Leveraging an empirical study, we will look at two dimension reduction techniques and how they can be applied to a Gaussian Processes. Figure 1: overview of the method. CNN is a convolutional neural net and GPR/VGPR are different Gaussian Process Regressions. Image by author. Regarding implementation (…)
The cheap way how to use Docker to deploy your FastAPI
FastAPI is a powerful API framework for Python that allows to quickly create and develop APIs in Python. But how to deploy those APIs? What is FastAPI? From the FastAPI website: FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints. What does this mean? It means that FastAPI allows to quickly build and develop APIs with (…)
Python Script to Send Emails. Code and explanation to send emails…
Code and explanation to send emails using Python. Emails are a typical method of official communication nowadays, and they are also useful for transferring files from one person to another. Almost everyone who has an online identity or merely utilizes it has their email address, whether it’s Gmail or Outlook. One of the many cool things you can do with Python is to send and receive emails. Python (…)
How to Boost 10X Productivity with Tmux
Personal Advice for Productive Data Scientists For Data Scientists and Software Engineers Tmux is a terminal multiplexer. That means you can view multiple terminal views and histories in one session (sourced by author) Problem Statement Oh no, I closed my terminal and lost my progress to run <ML model> Oh no, I lost track of my commands to deploy scripts I wish there were easy ways to keep (…)
How to tune multiple ML models with GridSearchCV at once?
Speed up your model selection workflow Image by LTD EHU from Pixabay Model selection is an essential component for a data science model development pipeline. After performing feature engineering the data scientist needs to choose the model with the best set of hyperparameters that performs best for the training dataset. There are various Auto-ML libraries that automate the model selection (…)
Wykrywanie anomalii – na czym polega i jak je wykrywać?
– Hej tato, pobawisz się z nami? – przybiegły z pytaniem Jagódka i Otylka. – Jeszcze chwilkę Księżniczki. Dokończę tylko jedną bardzo ważną rzecz i się pobawimy – odpowiedziałem. – A co lobisz tatusiu? – zapytała Otylka. – Buduję model do wykrywania anomalii w transakcjach, aby uratować naszych klientów przed … Artykuł Wykrywanie anomalii – na czym polega i jak je wykrywać? pochodzi z serwisu (…)
Zestawienie linków przygotowuje automat, wybacz więc wszelkie dziwactwa ;-)