#postgresSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

8 hours ago

Understanding Concurrency in PostgreSQL: How It Handles Multiple Transactions Efficiently Introduction Continue reading on Medium »

#sql #postgressql #design #software-architecture #software-development

Origin | Interest | Match

1 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

3 days ago

OpenAI’s Postgres Architecture: A Brilliant Fix for a Billion-Dollar Mistake Author: Matthew Penaroza, Head of AI Solution Architecture at SurrealDB OpenAI’s write-up on scaling...

OpenAI’s Postgres Architecture: A Brilliant Fix for a Billion-Dollar Mistake Author: Matthew Penaroza , Head of AI Solution Architecture at SurrealDB OpenAI’s write-up on scaling PostgreSQL for...

#database #openai #postgres #postgressql

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

4 days ago

SQL Joins & Window Functions - prodSens.live JOINS They allow us to work with multiple tables and allows us to join data in different tables.…

SQL Joins & Window Functions JOINS They allow us to work with multiple tables and allows us to join data in different tables.… The post SQL Joins & Window Functions appeared first on prod...

#Software #postgres #postgressql #prodsens #live #sql

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

4 days ago

SQL Joins & Window Functions - prodSens.live JOINS They allow us to work with multiple tables and allows us to join data in different tables.…

SQL Joins & Window Functions JOINS They allow us to work with multiple tables and allows us to join data in different tables.… The post SQL Joins & Window Functions appeared first on prod...

#Software #postgres #postgressql #prodsens #live #sql

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

4 days ago

SQL Joins & Window Functions JOINS They allow us to work with multiple tables and allows us to join data in different...

SQL Joins & Window Functions JOINS They allow us to work with multiple tables and allows us to join data in different tables. Joins happen when there's is a primary and foreign keys-they al...

#sql #postgres #postgressql

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

3 weeks ago

How I Built a System That Tracks 900,000 Real-Time Events Per Day - prodSens.live A few months ago I started building a system to collect and analyze real-time event data. What began…

How I Built a System That Tracks 900,000 Real-Time Events Per Day A few months ago I started building a system to collect and analyze real-time event data. What began… The post How I Built a Syst...

#Software #backend #database #postgressql #prodsens #live #webdev

Origin | Interest | Match

1 0 0 0

Josué Kouka

@josuebrunel.bsky.social

1 month ago

From the PostgreSQL community on Reddit: Scaling PostgreSQL to power 800 million ChatGPT users Posted by Ncell50 - 105 votes and 10 comments

www.reddit.com/r/PostgreSQL.... #PostgresSQL #OpenAI #Scaling #Database

1 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

4 months ago

🧱 Lesson 5 – Working with PostgreSQL (Multi-Database Setup) Series: From Code to Cloud: Building a Production-Ready .NET Application By: Farrukh Rehman - Senior .NET Full Stack Develope...

#Software #csharp #entityframework #postgres #postgressql #prodsens #live

Origin | Interest | Match

1 0 0 0

DEV Community

@dev.to

4 months ago

I Built a Multi-Agent Code Review System That's 4x Faster (Thanks to Zero-Copy Database Forks!) "I didn't know you could do that!" - My reaction when I discovered Tiger Cloud's zero-copy...

A little over a week left in the Agentic Postgres Challenge with Tiger Database. Need some inspiration? Check out this project by Simran Shaikh where she runs multiple AI agents concurrently for super speedy code analysis.

dev.to/simranshaikh...

#agenticpostgreschallenge #postgressql #ai

1 0 0 0

Ubuntu

@ubuntu.activitypub.awakari.com.ap.brid.gy

6 months ago

How to Configure Nagios to Monitor a PostgreSQL 16 Database with Dashboards on Ubuntu 24.04 LTS #postgresql #postgres #nagios inchirags@gmail.com Chirag PostgreSQL DBA Tutorial https://www.chirags.in How to...

How to Configure Nagios to Monitor a PostgreSQL 16 Database with Dashboards on Ubuntu 24.04 LTS #postgresql #postgres #nagios inchirags@gmail.com Chirag PostgreSQL DBA Tutorial https://www.chirags

#postgres #postgressql #nagios #database

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

6 months ago

How to Design a PostgreSQL Schema Visually (Step-by-Step) 1. What is a Schema? In PostgreSQL, a schema is just a folder inside your database where you… The post How to Design a PostgreSQL Schema ...

#Software #database #postgres #postgressql #prodsens #live #sql

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

6 months ago

Using PgBouncer to improve performance and reduce the load on PostgreSQL inchirags@gmail.com Chirag's PostgreSQL DBA Tutorial https://www.chirags.in Using PgBouncer to...

Using PgBouncer to improve performance and reduce the load on PostgreSQL inchirags@gmail.com Chirag's PostgreSQL DBA Tutorial https://www.chirags.in Using PgBouncer to improve performance and r...

#pgbouncer #postgres #postgressql #database

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

7 months ago

Manage Replication and Failover on a PostgreSQL 16 Cluster Using repmgr in 02 Nodes on Ubuntu 24.04 LTS inchirags@gmail.com Chirag's PostgreSQL DBA Tutorial ...

Manage Replication and Failover on a PostgreSQL 16 Cluster Using repmgr in 02 Nodes on Ubuntu 24.04 LTS inchirags@gmail.com Chirag's PostgreSQL DBA Tutorial https://www.chirags.in Manage Replic...

#repmgr #postgres #database #postgressql

Origin | Interest | Match

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

9 months ago

Building a smarter Web scraper: Vector embeddings for intelligent content retrieval and analysis # Yet Another Scraper? This One's Actually Worth Your Time! Hey there, fellow code wranglers! 👋 Alex here, and today I'm going to introduce you to my latest python creation - (and yes, I will do a rust follow-up if this gets enough attention). ## Foreword Whether you're just starting your coding journey or you've got a few years under your belt as a developer, this project is designed to be both educational and immediately useful. Entry-level devs will find the codebase approachable with clear patterns to learn from, while mid-level engineers can appreciate the architecture choices and extend the functionality for their specific needs - it is meant to be a starting point for you to build upon. ## What Makes This One Special? This isn't just any scraper. It's a full-stack solution that combines `FastAPI`, `PostgreSQL` with `pgvector`, and `Playwright` to create a powerful content scraping and similarity search system. Think of it as your personal web librarian that not only collects content but also helps you find related information using the magic of vector embeddings. User: "Find me content similar to this article about AI ethics" Scraper: *does vector magic* "Here are 5 related articles ranked by similarity" User: *surprised Pikachu face* ## The stack * **FastAPI** : Because life's too short for slow APIs * **PostgreSQL + pgvector** : Vector similarity search that doesn't make your database cry * **Playwright** : Headless browsing that actually works with modern websites * **Sentence Transformers** : Turning words into math (vectors) so computers can understand content similarity ## Show Me The Code Already! Here's how easy it is to scrape URLs: # The regular endpoint supports multiple URLs too curl -X POST http://localhost:8000/v1/scrape/ \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{"urls": ["https://dev.to/alexandrughinea", "https://dev.to/topics/python"]}' And searching for similar content is just as simple: curl -X GET "http://localhost:8000/v1/search?text=fastapi%20vector%20search&limit=5" \ -H "X-API-Key: your_api_key" For larger batches, the async batch endpoint provides better performance: # The batch endpoint processes URLs asynchronously for better performance curl -X POST http://localhost:8000/v1/scrape/batch/ \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{"urls": ["https://example.com", "https://example.org", "https://example.net"]}' And once you've scraped some content, you can query it directly: # Get all scraped data (paginated) curl -X GET "http://localhost:8000/v1/data/?limit=10" \ -H "X-API-Key: your_api_key" # Get a specific item by ID curl -X GET "http://localhost:8000/v1/data/42" \ -H "X-API-Key: your_api_key" ## The Secret Sauce: Vector Similarity The real magic happens in the search functionality. When you scrape content, it gets converted into vector embeddings (fancy math arrays) that represent the semantic meaning of the text. Then when you search, your query text gets converted to the same format, and we find the closest matches using cosine similarity. ## Why I Built This I got tired of scraping websites and then having to build separate systems for search and content analysis. This project combines everything into one cohesive system that: 1. Scrapes content efficiently using headless browsers 2. Stores content and vector embeddings in PostgreSQL 3. Provides a clean API for searching and retrieving similar content 4. Handles batch operations for processing multiple URLs (sync and async versions) 5. Avoids duplicate content through similarity detection 6. Respects robots.txt rules (configurable via environment variables) ## Getting Started in 30 Seconds # Clone the repo git clone https://github.com/alexandrughinea/python-fastapi-postgres-scraper # Start with Docker cd python-fastapi-postgres-scraper docker-compose up -d # Your API is now running at http://localhost:8000 ## Final Thoughts Is this yet another scraper? Technically, yes. But it's a scraper with superpowers. It doesn't just collect data; it understands it, organizes it, and helps you find connections between different pieces of content in your own database. The next time someone says "we need to scrape some websites," you can smugly pull out this repo instead of cobbling together a BeautifulSoup script that will break the moment the website changes its CSS classes. ## Getting Started To make your life even easier, I've included Bruno API documentation in the `/docs` folder. Bruno is a great open-source API client that lets you test all the endpoints without writing a single line of code. Just install Bruno, open the collection, and start experimenting! The complete project is available on GitHub at github.com/alexandrughinea/python-fastapi-postgres-scraper. Star it if you find it useful, and contributions are always welcome! Give it a try and let me know what you think! And remember, with great scraping power comes great responsibility, don't hammer websites with requests, be respectful of robots.txt, and maybe consider asking for permission first - because being a good web citizen matters. You can toggle this behavior with the `RESPECT_ROBOTS_TXT` environment variable if you really need to. Happy scraping! 🕸️🚀

Building a smarter Web scraper: Vector embeddings for intelligent content retrieval and analysis ...

dev.to/alexandrughinea/building...

#python #ai #postgressql #fastapi

Result Details

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

9 months ago

Building a smarter Web scraper: Vector embeddings for intelligent content retrieval and analysis # Yet Another Scraper? This One's Actually Worth Your Time! Hey there, fellow code wranglers! 👋 Alex here, and today I'm going to introduce you to my latest python creation - (and yes, I will do a rust follow-up if this gets enough attention). ## Foreword Whether you're just starting your coding journey or you've got a few years under your belt as a developer, this project is designed to be both educational and immediately useful. Entry-level devs will find the codebase approachable with clear patterns to learn from, while mid-level engineers can appreciate the architecture choices and extend the functionality for their specific needs - it is meant to be a starting point for you to build upon. ## What Makes This One Special? This isn't just any scraper. It's a full-stack solution that combines `FastAPI`, `PostgreSQL` with `pgvector`, and `Playwright` to create a powerful content scraping and similarity search system. Think of it as your personal web librarian that not only collects content but also helps you find related information using the magic of vector embeddings. User: "Find me content similar to this article about AI ethics" Scraper: *does vector magic* "Here are 5 related articles ranked by similarity" User: *surprised Pikachu face* ## The stack * **FastAPI** : Because life's too short for slow APIs * **PostgreSQL + pgvector** : Vector similarity search that doesn't make your database cry * **Playwright** : Headless browsing that actually works with modern websites * **Sentence Transformers** : Turning words into math (vectors) so computers can understand content similarity ## Show Me The Code Already! Here's how easy it is to scrape URLs: # The regular endpoint supports multiple URLs too curl -X POST http://localhost:8000/v1/scrape/ \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{"urls": ["https://dev.to/alexandrughinea", "https://dev.to/topics/python"]}' And searching for similar content is just as simple: curl -X GET "http://localhost:8000/v1/search?text=fastapi%20vector%20search&limit=5" \ -H "X-API-Key: your_api_key" For larger batches, the async batch endpoint provides better performance: # The batch endpoint processes URLs asynchronously for better performance curl -X POST http://localhost:8000/v1/scrape/batch/ \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{"urls": ["https://example.com", "https://example.org", "https://example.net"]}' And once you've scraped some content, you can query it directly: # Get all scraped data (paginated) curl -X GET "http://localhost:8000/v1/data/?limit=10" \ -H "X-API-Key: your_api_key" # Get a specific item by ID curl -X GET "http://localhost:8000/v1/data/42" \ -H "X-API-Key: your_api_key" ## The Secret Sauce: Vector Similarity The real magic happens in the search functionality. When you scrape content, it gets converted into vector embeddings (fancy math arrays) that represent the semantic meaning of the text. Then when you search, your query text gets converted to the same format, and we find the closest matches using cosine similarity. ## Why I Built This I got tired of scraping websites and then having to build separate systems for search and content analysis. This project combines everything into one cohesive system that: 1. Scrapes content efficiently using headless browsers 2. Stores content and vector embeddings in PostgreSQL 3. Provides a clean API for searching and retrieving similar content 4. Handles batch operations for processing multiple URLs (sync and async versions) 5. Avoids duplicate content through similarity detection 6. Respects robots.txt rules (configurable via environment variables) ## Getting Started in 30 Seconds # Clone the repo git clone https://github.com/alexandrughinea/python-fastapi-postgres-scraper # Start with Docker cd python-fastapi-postgres-scraper docker-compose up -d # Your API is now running at http://localhost:8000 ## Final Thoughts Is this yet another scraper? Technically, yes. But it's a scraper with superpowers. It doesn't just collect data; it understands it, organizes it, and helps you find connections between different pieces of content in your own database. The next time someone says "we need to scrape some websites," you can smugly pull out this repo instead of cobbling together a BeautifulSoup script that will break the moment the website changes its CSS classes. ## Getting Started To make your life even easier, I've included Bruno API documentation in the `/docs` folder. Bruno is a great open-source API client that lets you test all the endpoints without writing a single line of code. Just install Bruno, open the collection, and start experimenting! The complete project is available on GitHub at github.com/alexandrughinea/python-fastapi-postgres-scraper. Star it if you find it useful, and contributions are always welcome! Give it a try and let me know what you think! And remember, with great scraping power comes great responsibility, don't hammer websites with requests, be respectful of robots.txt, and maybe consider asking for permission first - because being a good web citizen matters. You can toggle this behavior with the `RESPECT_ROBOTS_TXT` environment variable if you really need to. Happy scraping! 🕸️🚀

Building a smarter Web scraper: Vector embeddings for intelligent content retrieval and analysis ...

dev.to/alexandrughinea/building...

#python #ai #postgressql #fastapi

Result Details

0 0 0 0

Forem

@forem.com.web.brid.gy

9 months ago

How to Set Up PostgreSQL for Your Django App on a VPS Server If you're hosting your first Django API on a server (like Contabo) and you’ve previously used PostgreSQL locally, you’ll need to configure it on your server too. This guide will walk you through installing and configuring PostgreSQL on your VPS and connecting it to your Django app using your `.env` file. ## Prerequisites * A running VPS (e.g., from Contabo) with Ubuntu or Debian. * Domain/subdomain already set up. * Django project already deployed to the server. * `.env` file with your database credentials ready. * SSH access to the server. ## Step 1: Install PostgreSQL on Your VPS SSH into your server: ssh your-user@your-server-ip Then install PostgreSQL and its dependencies: sudo apt update sudo apt install postgresql postgresql-contrib libpq-dev ## Step 2: Create PostgreSQL Database and User Switch to the `postgres` user: sudo -u postgres psql Run the following SQL commands to create a database and user (update placeholders to match your actual values from `.env`): CREATE DATABASE mydb_name; CREATE USER mydb_user WITH PASSWORD 'strongpassword'; ALTER ROLE mydb_user SET client_encoding TO 'utf8'; ALTER ROLE mydb_user SET default_transaction_isolation TO 'read committed'; ALTER ROLE mydb_user SET timezone TO 'UTC'; GRANT ALL PRIVILEGES ON DATABASE mydb_name TO mydb_user; \q ## Step 3: Configure PostgreSQL for Remote Access (Optional) Only do this **if** you need to connect to the DB remotely (e.g., from your local machine or external tools). ### Update `pg_hba.conf`: sudo nano /etc/postgresql/*/main/pg_hba.conf Add or update the line: host all all 0.0.0.0/0 md5 ### Update `postgresql.conf`: sudo nano /etc/postgresql/*/main/postgresql.conf Uncomment and edit: listen_addresses = '*' Restart PostgreSQL: sudo systemctl restart postgresql ## Step 4: Configure Django to Use PostgreSQL Update your `.env` file in the Django project root: DB_NAME=mydb_name DB_USER=mydb_user DB_PASSWORD=strongpassword DB_HOST=localhost DB_PORT=5432 Update your `settings.py` to use `os.environ` (if not done already): import os DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'NAME': os.getenv('DB_NAME'), 'USER': os.getenv('DB_USER'), 'PASSWORD': os.getenv('DB_PASSWORD'), 'HOST': os.getenv('DB_HOST'), 'PORT': os.getenv('DB_PORT', '5432'), } } ## Step 5: Install PostgreSQL Client for Django If you haven’t already installed it: pip install psycopg2-binary ## Step 6: Run Django Migrations Once the connection is set, apply migrations: python manage.py makemigrations python manage.py migrate You can run the development server to confirm it works: python manage.py runserver 0.0.0.0:8000 Then visit `http://your-server-ip:8000` to confirm everything runs as expected. ## Bonus: Secure PostgreSQL (Recommended) If you're not using external tools to access the DB: * Keep `DB_HOST=localhost` * Avoid opening port `5432` to the world (skip remote config steps) ## What’s Next? Once PostgreSQL is working with Django: * Set up **Gunicorn** and **Nginx** for production deployment if not earlier done. * Use **HTTPS** with Let’s Encrypt. * Set up **supervisor** to keep your Django app running.

0 0 0 0

DEV Community

@dev.to.web.brid.gy

9 months ago

Schemas in PostgreSQL: A Practical Guide for Developers Schemas in PostgreSQL aren’t just for large systems—they’re for anyone who wants to keep their data structured. A schema is like a folder within your database where related objects (tables, views, etc.) are grouped together. This helps you separate concerns, organize logic, and secure your data more effectively. Here’s a practical look at how they work. ## Why You Should Use Schemas * **Organization** : Separate business domains (like inventory and users). * **Control** : Limit access to different parts of your app. * **Maintenance** : Make backups and updates more targeted. * **Cleaner Queries** : Avoid clutter in your namespace. ## Schema Types * ### Public: Comes with every PostgreSQL database. If you don’t specify a schema, objects go here. * ### Custom: Created manually to isolate logic or control access. CREATE SCHEMA hr; **How Data is Structured** PostgreSQL Cluster └── Database └── Schema └── Tables, Views, Functions, etc. This layered model helps manage growing projects without chaos. ## Real Example: E-commerce Separation CREATE SCHEMA inventory; CREATE TABLE inventory.products ( product_id serial PRIMARY KEY, product_name VARCHAR(255), price DECIMAL, stock_quantity INT ); CREATE TABLE public.users ( user_id serial PRIMARY KEY, username TEXT, email TEXT ); Using schemas lets you organize your app by responsibility—making it easier to evolve or scale parts of your system. ## FAQs ### Can I have multiple schemas in one DB? Yes, and it’s a common best practice. ### What’s the difference between public and custom schemas? Public is open by default; custom is restricted unless granted. ### Can schemas improve performance? Not directly, but they help manage large systems more cleanly. ### Are schemas portable between environments? Yes, using tools like `pg_dump` or schema migration scripts. ## Conclusion PostgreSQL schemas give you better structure, access control, and scalability. They're an essential tool for organizing your data without complicating your architecture. For a more detailed guide, visit the schemas in PostgreSQL article.

0 0 0 0

PostgreSQL

@postgresql.activitypub.awakari.com.ap.brid.gy

10 months ago

[Boost] Implementing Distributed Caching with PostgreSQL in .NET:...

[Boost] Implementing Distributed Caching with PostgreSQL in .NET: Sats.PostgresDistributedCache S...

https://dev.to/gentilpinto/-2n2c

#distributedcaching #postgressql #dotnet #caching

Result Details

0 0 0 0

LLMs

@llms.activitypub.awakari.com.ap.brid.gy

10 months ago

Supabase’s $200M Raise Signals Big Ambitions Supabase has announced a $200 million Series D fun...

www.bigdatawire.com/2025/04/24/supabases-200...

#News #in #Brief #backend #coatue #developer #Firebase #funding #open #source #PostgresSQL

Result Details

1 0 0 0

Oskar Dudycz

@oskardudycz.bsky.social

10 months ago

Let's build event store in one hour! - Oskar Dudycz - NDC Oslo 2022 YouTube video by NDC Conferences

I’m probably also one to blame as I showed how to build #PostgresSQL event store in one hour: m.youtube.com/watch?v=gaoZ...

Tho, in the end I concluded with “Kids don't do this at home!” warning. But then we know how many people are watching something till the end 🙃

1 0 1 0

DEV Community

@dev.to.web.brid.gy

11 months ago