#DataPreprocessing

BIFOLD Berlin Institute for the Foundations of Learning and Data

@bifold.berlin

4 days ago

BIFOLD is hiring: #Phd Candidate. Focus: Data Science & AI.

Details
💼 D2IP Group, led by Ziawasch Abedjan
🗓 Start date: May 6, 2026
📅 Apply by: March 27, 2026
🔗https://t1p.de/dfhtm

@tuberlin.bsky.social #jobvacancy
#jobalert #ScienceJobs #DataIntegration #DataPreprocessing #jobvacancy

Proceedings Series MDPI

@proceedingsmdpi.bsky.social

1 week ago

Effect of Data Preparation on Machine Learning Models for Diabetes Prediction
www.mdpi.com/2673-4591/12...

By Goran Martinović et al.
From the 34th International Scientific Conference on Organization and Technology of Maintenance

#MachineLearning #DataPreprocessing

EkasCloud – Personalized Training Platform

@ekascloud.bsky.social

1 month ago

Post from EkasCloud Online Courses - YouTube Feature Engineering Best Practices A Guide for Data Scientists #FeatureEngineering #DataScience #MachineLearning #MLModels #DataPreprocessing #AI #DataScient...

Feature Engineering Best Practices A Guide for Data Scientists
www.youtube.com/post/UgkxvhZ...
#FeatureEngineering #DataScience #MachineLearning #MLModels #DataPreprocessing #AI #DataScientists #ModelAccuracy #DataTransformation #TechInsights #Analytics #BigData #MLBestPractices #EkasCloud

EkasCloud – Personalized Training Platform

@ekascloud.bsky.social

1 month ago

Feature Engineering Best Practices: A Guide for Data Scientists
#FeatureEngineering #DataScience #MachineLearning #MLBestPractices #DataPreparation #DataPreprocessing #AI #ModelBuilding #BigData #Analytics #DataEngineering #MLModels #TechInsights #EkasCloud

EkasCloud – Personalized Training Platform

@ekascloud.bsky.social

1 month ago

Feature Engineering Best Practices A Guide for Data Scientists Feature engineering is a cornerstone of effective machine learning. By transforming raw data into meaningful features, data scientists can improve model performance, interpretability, and generaliz...

Feature Engineering Best Practices A Guide for Data Scientists
www.ekascloud.com/our-blog/fea...
#FeatureEngineering #DataScience #MachineLearning #MLModels #DataPreprocessing #AI #DataScientists #ModelAccuracy #DataTransformation #TechInsights #Analytics #BigData #MLBestPractices #EkasCloud

@arxivlens.bsky.social

3 months ago

An accessible approach to density estimation neural networks with data preprocessing.
Hou, Bosi et al.
Paper
Details
#DensityEstimation #NeuralNetworks #DataPreprocessing

JMIR Publications

@jmirpub.bsky.social

4 months ago

Preprocessing Large-Scale Conversational Datasets: A Framework and Its Application to Behavioral Health Transcripts Background: The rise of AI and accessible audio equipment has led to a proliferation of recorded conversation transcripts datasets across various fields. However, automatic mass recording and transcription often produce noisy, unstructured data. First, these datasets naturally include unintended recordings, such as hallway conversations, background noise and media (e.g., TV programs, radio, phone calls). Second, automatic speech recognition (ASR) and speaker diarization errors can result in misidentified words, speaker misattributions, and other transcription inaccuracies. As a result, large conversational transcript datasets require careful preprocessing and filtering to ensure their research utility. This challenge is particularly relevant in behavioral health contexts (e.g., therapy, treatment, counselling): while these transcripts offer valuable insights into patient-provider interactions, therapeutic techniques, and client progress, they must accurately represent the conversations to support meaningful research. Objective: We present a framework for preprocessing and filtering large datasets of conversational transcripts and apply it to a dataset of behavioral health transcripts from community mental health clinics across the United States. Within this framework we explore tools to efficiently filter non-sessions – transcripts of recordings in these clinics that do not reflect a behavioral treatment session but instead capture unrelated conversations or background noise. Methods: Our framework integrates basic feature extraction, human annotation, and advanced applications of large language models (LLMs). We begin by mapping transcription errors and assessing the distribution of sessions and non-sessions. Next, we identify key features to analyze how outliers help in characterizing the type of transcript. Notably, we use LLM perplexity as a measure of comprehensibility to assess transcript noise levels. Finally, we use zero-shot LLM prompting to classify transcripts as sessions or non-sessions, validating LLM decisions against expert annotations. Throughout, we prioritize data security by selecting tools that preserve anonymity and minimize the risk of data breaches. Results: Our findings demonstrated that basic statistical outliers, such as speaking rate, are associated with transcription errors and are observed more frequently in non-sessions versus sessions. Specifically, LLM perplexity can flag fragmented and non-verbal segments and is generally lower in sessions (permutation test mean difference = -258, p

JMIR Formative Res: Preprocessing Large-Scale Conversational Datasets: A Framework and Its Application to Behavioral Health Transcripts #AI #DataScience #MachineLearning #SpeechRecognition #DataPreprocessing

FreelanceBar.me

@freelancebarme.bsky.social

9 months ago

https://machinelearningmastery.com/10-python-one-liners-for-feature-selection-like-a-pro/

Data preprocessing in data analysis and machine learning is crucial for accurate results and better models #DataPreprocessing #MachineLearning machinelearningmastery.com/10-python-one-liners-for...