#INT4

Latest posts tagged with #INT4 on Bluesky

Trending

#Wildflower Hour #Misunderstood Songs #Firefly #Canada's Politics #Spurs #Oscars #Liverpool FC #Ukraine Conflict #U.S. Foreign Policy #F1 #Wildflower Hour #Misunderstood Songs #Firefly #Canada's Politics #Spurs #Oscars #Liverpool FC #Ukraine Conflict #U.S. Foreign Policy #F1

Posts tagged #INT4

@doyouknnow.bsky.social

3 months ago

LLM 양자화 완벽 가이드! INT4로 메모리 87.5% 절감, FP8로 처리량 43% 향상. GPTQ vs AWQ vs GGUF 비교, Llama 3 양자화 성능 벤치마크, Q4까지 손실 2% 미만! Pruning + Knowledge Distillation 경량화 기법, 하드웨어별 추천 전략, QLoRA Fine-tuning까지!

#AWQ #FP8 #GGUF #GPTQ #INT4 #INT8 #KnowledgeDistillation #Llama3 #llamacpp
doyouknow.kr/618/llm-quan...

0 0 0 0

ZimaSpace

@zimaspace.bsky.social

9 months ago

🔥 30 tokens/s on DeepSeek R1 671B (INT4) — no cluster, just one GPU

🖥️Set up:
· Dual AMD EPYC 9005 CPUs
· 1× RTX 4090 48GB
· DDR5 6400 1TB RAM

This is what local LLM power looks like.

#LocalAI #DeepSeek #MoE #INT4 #LLM #RTX4090 #EPYC #AIinference #EdgeAI #RAG