#ChatGLM #GLM130B #CodeGeeX #CogVLM #CogView #AMiner The Knowledge Engineering Group (KEG) and THUDM at @Tsinghua_Uni @jietang @ericdongyx
- Tweets 238
- Following 169
- Followers 4,271
- Likes 289
Pinned Tweet
#VisualAgentBench: 4o, 4o-mini, 3.5-sonnet currently have an edge as visual foundation agents for now, but open models InternVL & GLM-4V are catching up fast, a similar story to LLMs as agents as revealed in #AgentBench back in Aug 2023.
arxiv.org/pdf/2408.06327
github.com/THUDM/VisualAgent…
2
6
26
8,863
Introducing GLM-130B, an open bilingual (English&Chinese) language model with 130 billion parameters, designed for users with a single A100 (40G*8) or V100 (32G*8) server. #GLM130 #LLM #GPT3
Learn more & download:
keg.cs.tsinghua.edu.cn/glm-1…
github.com/THUDM/GLM-130B
11
67
374
ChatGLM-6B & ChatGLM!
ChatGLM-6B is an open CN&EN model w/ 6.2B paras (optimized for Chinese QA & dialogue for now). Trained for 1T tokens, SFT, Feedback Bootstrap, & RLHF.
w INT4 quantization, we can deploy on one 2080Ti card (6GB GPU mem required).
github.com/THUDM/ChatGLM-6B/…
13
78
276
110,199
Introducing #CodeGeeX, an open 13B multilingual code generative model capable of 1) generation of 15 programming languages & 2) translating btw on one click!
Learn more: keg.cs.tsinghua.edu.cn/codeg…
Search "codegeex" in #VSCode FOR FREE!
DEMO: models.aminer.cn/codegeex/pl…
Results next:
11
55
250
#CogVLM: open vision language models - deep fusion btn LLM & image encoder w/ visual experts!
CogVLM-17B tops 14 cross-modal benchs, beats #BLIP2, #PaLI-17B/X-55B, #PaLM-E-84B.
Paper: github.com/THUDM/CogVLM
HF🤗: huggingface.co/THUDM/CogVLM/…
@_akhaliq @osanseviero @huggingface
3
66
243
81,577
GLM-130B reaches INT4 quantization w/ no perf degradation, allowing effective inference on 4*3090 or 8*2080 Ti GPUs, the most ever affordable GPUs required for using 100B-scale models?
Paper: arxiv.org/abs/2210.02414
Model weights & code & demo & lessons: github.com/THUDM/GLM-130B
5
40
235
ChatGLM-6B is on the top spot of the 🤗 7-day trending? 220K downloads in a few days, and counting! A big thank you to @huggingface @_akhaliq @osanseviero @yvrjsharma @ClementDelangue and everyone!
HF: huggingface.co/THUDM/chatglm…
GH: github.com/THUDM/ChatGLM-6B
1
41
213
49,682
LoRA & P-Tuning-v2 & other supports added for ChatGLM-6B. Much fewer resources required for tuning your own models! The open-source community is the lead! Check out those efforts here: github.com/THUDM/ChatGLM-6B/…
3
57
192
39,264
#CodeGeeX2-6B: 2nd gen. code generation model for 100+ lang. Free #VSCode & #JetBrains plugins (10M lines gen-ed per day). It is based on #ChatGLM2-6B with extra 600B code training. huggingface.co/THUDM/codegee…
@huggingface @_akhaliq @osanseviero @ClementDelangue @code @kdd_news
2
39
142
46,988
Thanks!
The model weights & source code are freely available at keg.cs.tsinghua.edu.cn/codeg…
Search "codegeex" in #VSCode FOR FREE!!
The @Gradio Demo for CodeGeeX, a large-scale multilingual code generation model with 13 billion parameters, pre-trained on a large code corpus of more than 20 programming languages is now on @huggingface Spaces
demo: huggingface.co/spaces/THUDM/…
3
32
145
How to #RLHF for LLMs: #PPO or #DPO?
Introducing #BPO (black-box prompt optimization) to align LLMs without model training.
1) ChatGPT + BPO > ChatGPT
2) GPT-4 + BPO > GPT-4
3) Vicuna + BPO > Vicuna + PPO/DPO
4) Vicuna + DPO + BPO > Vicuna + DPO
arxiv.org/pdf/2311.04155v1.p…
1
41
153
30,493
CogVLM: vision language models w/ visual experts that tops 14 cross-modal benchmarks. Paper, model, code
@HuggingFace🤗: huggingface.co/THUDM/CogVLM/…
List of contemporary GPT4V-like open source models
- Blip2
- Llava1.5
- CogVLM by @thukeg
- StableVL by @vilm_hq (not out yet)
- @skunkworks_ai 's model (not out yet)
Did I miss any?
1
21
138
44,290
OPEN #ChatGLM3-6B: the 3rd gen
1) tops 44 tasks among <10B models
2) supports tool/function call, code interpreter, agent tasks, 32K
github.com/THUDM/ChatGLM3
🤗huggingface.co/THUDM/chatglm…
#ChatGLM-6Bs: 10M 🤗downloads, thank YOU! @huggingface @ClementDelangue @_akhaliq @osanseviero
3
25
119
47,084
#ChatGLM-6B v1.1: improved *English* dialogue & translation ability, in addition to Chinese.
=>
#VisualGLM-6B: a multimodal conversational language model that supports 1) images, 2) Chinese, and 3) English w/ #ChatGLM-6B & BLIP2-Qformer.
github.com/THUDM/ChatGLM-6B
3
28
84
31,950
#ChatGLM2-6B-32K: better understands long texts based on the ChatGLM2-6B, up to 32K context length based on Positional Interpolation, trained with a 32K context length during dialogue alignment.
Download🤗 @huggingface
huggingface.co/THUDM/chatglm…
github.com/THUDM/ChatGLM2-6B @_akhaliq
15
65
16,650
Thanks AK!
#CogVLM: an #open visual language model. Chat w CogVLM about images @Gradio github.com/THUDM/CogVLM
CogVLM-17B tops 10+ benches: NoCaps, Flicker30k, RefCOCO, RefCOCO+, RefCOCOg, Visual7W, GQA, SciQA, VizWiz VQA & TDIUC, surpassing or matching PaLI-X 55B.
CogVLM: Visual Expert for Pretrained Language Models
paper page: huggingface.co/papers/2311.0…
introduce CogVLM, a powerful open-source visual language foundation model. Different from the popular shallow alignment method which maps image features into the input space of language model, CogVLM bridges the gap between the frozen pretrained language model and image encoder by a trainable visual expert module in the attention and FFN layers. As a result, CogVLM enables deep fusion of vision language features without sacrificing any performance on NLP tasks. CogVLM-17B achieves state-of-the-art performance on 10 classic cross-modal benchmarks, including NoCaps, Flicker30k captioning, RefCOCO, RefCOCO+, RefCOCOg, Visual7W, GQA, ScienceQA, VizWiz VQA and TDIUC, and ranks the 2nd on VQAv2, OKVQA, TextVQA, COCO captioning, etc., surpassing or matching PaLI-X 55B.
1
12
62
35,547
Lots of fun when you survived a tornado! The team enjoyed it a lot after training the GLM-130B.
Technical & engineering details and lessons learned when training 100B-scale models are in the ICLR'23 paper:
arxiv.org/pdf/2210.02414.pdf
3
7
48
2,989
The open ChatGLM-6B model (1T tokens pretraining + SFT + RLHF) is on the top spot of today's GitHub trending repos. Thank you!
Download & play the model here:
github.com/THUDM/ChatGLM-6B
7
43
3,571
#CogAgent: an open vision language model #VLM for building GUI agents, supporting 1120*1120 input & topping VQAv2, OK-VQ, TextVQA, ST-VQA, ChartQA, infoVQA, DocVQA, MM-Vet, POPE & surpassing models on GUI operation datasets including AITW and Mind2Web.
Paper: arxiv.org/pdf/2312.08914.pdf
Model & SFT dataset: github.com/THUDM/CogVLM
2
10
45
9,804
#WebGLM: the new member of the GLM family (GLM-130B, ChatGLM-130B, ChatGLM-6B, VisualGLM-6B). It is a web-enhanced QA system based on the General Language Model (GLM). All we need next is a search engine. github.com/THUDM/WebGLM
Paper: arxiv.org/pdf/2306.07906.pdf #KDD2023 @_akhaliq
10
38
9,330
Our #ChatGLM chief @jietang and team @ShawLiu12 @xujz0703 and many are at #NeurIPS2023. Also a proud sponsor of NeurIPS this year. See you there!
github.com/THUDM/ChatGLM3
I'm moderating a plenary panel at #NeurIPS2023 entitled "LLMs: Beyond Scaling" with some amazing researchers.
Please send or upvote any interesting questions:
dory.app/events/2KZxWFPULUn9…
3
32
8,508
Github: github.com/THUDM/CodeGeeX2
HF : huggingface.co/THUDM/codegee…
#CodeGeeX2-6B: 2nd gen. code generation model for 100+ lang. Free #VSCode & #JetBrains plugins (10M lines gen-ed per day). It is based on #ChatGLM2-6B with extra 600B code training. huggingface.co/THUDM/codegee…
@huggingface @_akhaliq @osanseviero @ClementDelangue @code @kdd_news
1
1
24
13,629
What a day! ChatGLM-6B & ChatGLM should have picked up another day for introduction on Twitter...🤣
2
1
24
3,754
The center of the AI universe 🪐 thank you for everything you’ve done for the community, incredible!
2
2
21
17,459
“Why is the sky blue?” See how #WebGLM explains it piped.video/watch?v=ohjrlYCL…
#WebGLM is a web-enhanced QA system based on the General Language Model (GLM). All we need next is a search engine. huggingface.co/THUDM/WebGLM 🤗github.com/THUDM/WebGLM
@kdd_news @_akhaliq #KDD2023
WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences
paper page: huggingface.co/papers/2306.0…
present WebGLM, a web-enhanced question-answering system based on the General Language Model (GLM). Its goal is to augment a pre-trained large language model (LLM) with web search and retrieval capabilities while being efficient for real-world deployments. To achieve this, we develop WebGLM with strategies for the LLM-augmented retriever, bootstrapped generator, and human preference-aware scorer. Specifically, we identify and address the limitations of WebGPT (OpenAI), through which WebGLM is enabled with accuracy, efficiency, and cost-effectiveness advantages. In addition, we propose systematic criteria for evaluating web-enhanced QA systems. We conduct multi-dimensional human evaluation and quantitative ablation studies, which suggest the outperformance of the proposed WebGLM designs over existing systems. WebGLM with the 10-billion-parameter GLM (10B) is shown to perform better than the similar-sized WebGPT (13B) and even comparably to WebGPT (175B) in human evaluation.
7
20
14,457
The paper is here: arxiv.org/abs/2303.17568
The @Gradio Demo for CodeGeeX, a large-scale multilingual code generation model with 13 billion parameters, pre-trained on a large code corpus of more than 20 programming languages is now on @huggingface Spaces
demo: huggingface.co/spaces/THUDM/…
2
17
2,079
ChatGLM-6B (1T tokens, SFT,RLHF) freely download here github.com/THUDM/ChatGLM-6B
The model is bilingual (Chinese & English ) but sft is optimized for Chinese for now. Looking to see how we can learn from #alpaca on English.
ChatGLM.cn based on glm-130b is online!
Models announced today:
- @OpenAI GPT4
- @AnthropicAI Claude
- @thukeg ChatGLM
- @GoogleAI Med-PaLM 2
Any more I need to add to foundationmodeltracker.com?
1
2
17
1,951
GLM-130B attempts to do this w/ completely open code, data, and all issues faced and lessons learned. So it could be a/the possible answer? github.com/THUDM/GLM-130B
1
1
18
ImageReward: an open general-purpose text-to-image human preference Reward Model & | a t2i scoring metric 30%+ over CLIP, BLIP, & Aesthetic in terms of human preferences.
github.com/THUDM/ImageReward
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
abs: arxiv.org/abs/2304.05977
github: github.com/THUDM/ImageReward
2
3
17
2,185
ChatGLM22222222-6B tops the trending 🙏
Trending models, datasets and spaces of the week, now at hf.co/models, hf.co/datasets & hf.co/spaces with all the filters by tasks, languages, licenses and more!
Congrats to @salesforce @OpenChatCo @erhartford @MosaicML @Replit @fkadev @MetaAI @TIIuae and many others!
3
16
4,713
#CogAgent Here are the HF links for downloading the models: Chat version🤗: huggingface.co/THUDM/cogagen…
VQA version🤗: huggingface.co/THUDM/cogagen…
#CogAgent: an open vision language model #VLM for building GUI agents, supporting 1120*1120 input & topping VQAv2, OK-VQ, TextVQA, ST-VQA, ChartQA, infoVQA, DocVQA, MM-Vet, POPE & surpassing models on GUI operation datasets including AITW and Mind2Web.
Paper: arxiv.org/pdf/2312.08914.pdf
Model & SFT dataset: github.com/THUDM/CogVLM
4
15
2,082
Our #CodeGeeX (13B) code generation & translation model is serving programmers on VS Code, JetBrains, and CloudStudio (Tencent's cloud IDE).
And it is FREE! Here is its performance among multilingual code models (evaluated on Sep. 2022):
CodeGeeX is an AI-powered code generation tool designed to accelerate your coding process. Similar to GitHub Copilot, but CodeGeeX is available for free.
medium.com/@innovation64feng…
@sama @elonmusk @liuren @zibuyu9 @liujiang @BillGates @csdncto
2
2
16
2,190
A brilliant #ChatGLM application scenario: Chat with Books or Wikipedia.
github.com/THUDM/ChatGLM-6B
用本地的 #ChatGLM 配合看书!
修改了下之前写的用ChatGPT辅助读书写的一套代码, 试着集成了@thukeg的开源版LLM,
可以基于书的内容的进行长问答+显示出处与章节.
话说他们最近还出了embedding, 还没来得及用.
期待更多中文作为第一公民的LLM了.
1
2
15
2,276
ChatGLM (chatglm.cn), the chat model based on GLM-130B, has been under alpha test w/ invited only-users since Feb. 28. See details here chatglm.cn/blog (in Chinese)
1
15
2,226
March 14: ChatGLM-6B 😉
github.com/THUDM/ChatGLM-6B/…
huggingface.co/THUDM/chatglm…
May 18: VisualGLM-6B 😄
huggingface.co/THUDM/visualg…
github.com/THUDM/VisualGLM-6…
May is not over, but so many exciting things this month! (2 days to surprise us!)
🔥QLoRA: 4-bit finetuning
🌸StarCoder and StarChat, SOTA Open Source Code models
🔊5x faster Whisper with less than 8GB GPU
Check them out and add any missing! ⤵️
github.com/osanseviero/ml_ti…
1
14
2,435
New protein & antibody #LLMs in the GLM family:
#xTrimoPGLM-100B: protein understanding & generation. SOTA in 13 of 15 tasks.
#xTrimoPGLM-Ab-1B: for antibodies, outperforming AF2 in structure prediction, 4000x inference speedup.
Paper: biorxiv.org/content/10.1101/…
@_akhaliq
4
11
4,807
Steaming output. ChatGLM-6B!
🚨🤖 Exciting news! Check out our new @Gradio Chatbot demo for ChatGLM-6B to create a bilingual chatbot with Streaming capabilities! @thukeg
[Video]:🤩With Gradio's new 🎨Theme feature, you can customize the chatbot to your liking just as I've tried adding THUDM colors🟪to the mix! We have a Themes hackathon underway too 🏆 JOIN- huggingface.co/Gradio-Themes
💬 ChatGLM-6B answers align with human preference. The model outputs are streamed directly to the chatbot component for seamless interaction. So, get ready to chat away with AI on Gradio! 🙌🚀
Demo on @huggingface - huggingface.co/spaces/ysharm…
1
3
12
2,852
#AgentTuning: Enabling Generalized Agent Abilities For LLMs, e.g., let #Llama2-7/13/70B to achieve 3.5-level agent abilities while stay good on MMLU, GSM8K, HumanEval. All models were behind GPT-4/3.5, Claude on #AgentBench.
🤗huggingface.co/THUDM/agentlm…
github.com/THUDM/AgentTuning
5
12
4,113
Thanks @Tim_Dettmers!! Super work on book and opt etc. It’d be great to chat on further pushing the boundary to make 100B-scale models as “small” as possible!
GLM-130B is excellent concurrent work with our LLM.int8(). It is wonderful to see how much you can learn by studying both side-by-side — each enhanced by others' insights. I can confirm many of the findings, e.g. scaling laws, but also have immediately new ideas to improve it!
2
11
Thanks @_akhaliq & his team for helping set up the @huggingface space for KEG: huggingface.co/thudm
GLM-130B, CogView2, CogVideo are included!!
A @Gradio Demo for GLM-130B: An Open Bilingual Pre-Trained Model on @huggingface Spaces by huggingface.co/hanyullai
demo: huggingface.co/spaces/hanyul…
Get started with Gradio: gradio.app/getting_started/
2
11
Another examples on Programming, world_knowledge, Referring Expression Comprehension (REC), and vqa
#CogVLM: open vision language models - deep fusion btn LLM & image encoder w/ visual experts!
CogVLM-17B tops 14 cross-modal benchs, beats #BLIP2, #PaLI-17B/X-55B, #PaLM-E-84B.
Paper: github.com/THUDM/CogVLM
HF🤗: huggingface.co/THUDM/CogVLM/…
@_akhaliq @osanseviero @huggingface
2
11
1,840
#AgentBench
🤯Static NLP datasets are not enough for evaluating existing #LLMs
🌟We should test them in practical interactive environments for agents!
Find more videos for LLM-as-Agent in #AgentBench at llmbench.ai
Thanks @arankomatsuzaki for sharing our paper #AgentBench !
🤯Static NLP datasets are not enough for evaluating existing LLMs
🌟We should test them in practical interactive environments for agents!
Find more videos for LLM-as-Agent in AgentBench at llmbench.ai !
1
1
10
1,978
Introducing kgTransformer at #KDD2022. It is designed to pre-train transformers on massive knowledge graphs with Mixture-of-Experts (MoE). It can answer complex logical queries in a unified manner. @kdd_news
Kudos to @ShawLiu12, Shiyu, Kai & all!
code: github.com/THUDM/kgTransform…
4
9
Thanks a million @LoubnaBenAllal1! You guys from @huggingface have been always so nice to help with this (immediately)!! Last time, @_akhaliq @osanseviero helped a lot on GLM-130B and our team space huggingface.co/thudm. Thank you!
Great work! We added HumanEval-X to the Hugging Face hub and we can transfer it to your HF organization huggingface.co/datasets/loub…. It would be great to have the models there too!
1
3
10
We are making #CodeGeeX freely available on VS Code (and with extra features such as code translation btw programming languages).
So like this kinda thing might be a Copilot competitor? @b_antunes
3
1
9
Excited to introduce XDAI #KDD2022, a toolkit for exploiting pretrained #LLM in knowledge-grounded dialogue generation without any training or fine-tuning. #GLM130B @kdd_news
Try XDAI ChatBot (Chinese Only for now): models.aminer.cn/xdai/
Developers: github.com/THUDM/XDAI
2
9
😅 thanks for the feedback! Lots of work to do :)
The best open-source chatbot from 🇨🇳 is now available to play with: ChatGLM-6B from Tsinghua's @thukeg.
Try it out: huggingface.co/spaces/ysharm…
It's impressive on many fronts, and displays many of the same weaknesses as other LLMs. Also does some interesting English-Chinese mixing👇
2
1
9
2,819
Jiezhong Qiu, a KEG alum at Tsinghua CS, wins the KDD'22 Outstanding Dissertation Award RunnerUp for his work on Graph Rep Learning. First time this award goes to a group in Asia since established in 2008! Congrats Jiezhong & his advisor @jietang @Tsinghua_Uni @kdd_news #KDD2022
2
7
Wow what a great lineup of speakers: CoT, GPT-4, Llama2, ChatGLM2, PaLM2!
The hottest topic of the premier conference! Please attend the LLM day at KDD-2023 next week! August 8, Los Angeles. bigmodel.ai/llmday-kdd23/
7
998
YES!! The current plan is to release #CodeGeeX model weights next week! Stay tuned ;)
7
Btw, our CogVLM arxiv submission (ArXiv ID 5148899) has been "on hold" for about two weeks without clear reasons. Is arXiv supposed to be a timely "publishing" model? Please help if possible @arxiv @_akhaliq 😖
#CogVLM: open vision language models - deep fusion btn LLM & image encoder w/ visual experts!
CogVLM-17B tops 14 cross-modal benchs, beats #BLIP2, #PaLI-17B/X-55B, #PaLM-E-84B.
Paper: github.com/THUDM/CogVLM
HF🤗: huggingface.co/THUDM/CogVLM/…
@_akhaliq @osanseviero @huggingface
1
7
1,552
#CodeGeeX open model (and paper) at github.com/THUDM/CodeGeeX
3. Replit reveals replit-code-v1-3b
Replit introduced their very own open-sourced LLaMa style LLM, 'replit-code-v1-3b.'
It's trained on 2.7 billion parameters and performs 40% better than comparable models.
2
6
1,393
Thanks for cooking the video! Proud to push the open sourced advances in LLMs!
Exploring the concept of a GLM (General Language Model) and working with ChatGLM6B.
ChatGLM6B is a 6.2B parm LLM, similar to ChatGPT, that can run on small as 6GB of memory.
Video: piped.video/watch?v=fGpXj4bl…
6
3,294
Cool, thanks for sharing!! GLM-130B was an attempt to open-source a 100B-scale model at least as good as GPT-3 (davinci) and unveil how models of such a scale can be successfully pre-trained! Stay tuned for more advanced models.
Btn GPT3 and ChatGPT&3.5, there was a long way...
4
528
The KEG team's paper on AMiner.cn wins the SIGKDD Test of Time Award!
Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang and Zhong Su received the Test of Time Award for Applied Science in recognition of their study of mining academic social networks published in the 2008 paper, "ArnetMiner: Extraction And Mining Of Academic Social Networks."
6
WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences
10B WebGLM performs better than 13B WebGPT and even comparably to 175B WebGPT in human evaluation.
The code, demo, and data are released.
repo: github.com/THUDM/WebGLM
abs: arxiv.org/abs/2306.07906
1
5
1,598
Here are more examples on OCR_free_reasoning, OCR_free_vqa, grounding_vqa, grounding_with_caption.
#CogVLM: open vision language models - deep fusion btn LLM & image encoder w/ visual experts!
CogVLM-17B tops 14 cross-modal benchs, beats #BLIP2, #PaLI-17B/X-55B, #PaLM-E-84B.
Paper: github.com/THUDM/CogVLM
HF🤗: huggingface.co/THUDM/CogVLM/…
@_akhaliq @osanseviero @huggingface
1
5
912
And @thukeg 😄
Trending models, datasets and spaces of the week on hf.co. Congrats to @cerebras @databricks @BananaDev_ (April Fool) @AnthropicAI @nomic_ai @Picsart @AlibabaGroup (excited about text to video!!) and all others!
1
5
2,019
There is a lot of work happening from davinci (GPT-3) to code-davinci-002 to text-davinci-002 (InstructGPT) to text-davinci-003 and beyond. The (currently) open-sourced GLM-130B is only on its first step. The mountain has many paths...but no easy one.
Maybe this question is too hard for a LLM without RLHF. Neither davinci nor text-davinci-002 can do it, only text-davinci-003 and chatgpt with RLHF can answer correctly
4
825
Big congrats! @ShawLiu12 from our team and Huan'ang!
We are thrilled to announce that Gao Huan'ang and Liu Xiao from #Tsinghua #TsinghuaDCST have won the 2023 Tsinghua University Top Student Award! This is the 5th year in a row that our department has excelled in both undergraduate and graduate levels. Congrats! 🎉🎉🎉
5
910
Well said. In addition, GLM-130B was an attempt to open-source a 100B-scale model at least as good as GPT-3 (**davinci**) and unveil how models of such a scale can be successfully pre-trained!
Btn GPT3(2020.05) and ChatGPT (2022.11)&3.5, there was a long way...Stay tuned🤗
4
768
looking for OpenAI-4V alternatives?
- LLaVA
- BakLLaVA
- CogVLM
- Qwen-VL
different tasks:
- VQA - answering questions about images
- OCR
- reading text - zero-shot detections
link: blog.roboflow.com/gpt-4-visi…
4
888
FYI, this model was trained from scratch (1T tokens + SFT&RLHF). It is the small version of ChatGLM (GLM-130B + SFT&RLHF) github.com/THUDM/GLM-130B
2
4
120
wow, proud that we @thukeg (thudm) from @thudcst @Tsinghua_Uni are among the very top. Also thanks @huggingface 🤗
github.com/THUDM
I was curious about which universities are using Hugging Face. The answer: over 5000 groups 🤯
Explore all universities at huggingface.co/spaces/osanse…
Some groups with the most likes: @thukeg, @HelsinkiNLP, @humphrey_shi Labs, @UKPLab, @uwnlp, and @stanfordnlp 🔥
1
3
1,613
Thanks AK! GLM -> GLM-130B -> ChatGLM-6B/130B -> VisualGLM-6B -> #WebGLM -> ?
WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences
paper page: huggingface.co/papers/2306.0…
present WebGLM, a web-enhanced question-answering system based on the General Language Model (GLM). Its goal is to augment a pre-trained large language model (LLM) with web search and retrieval capabilities while being efficient for real-world deployments. To achieve this, we develop WebGLM with strategies for the LLM-augmented retriever, bootstrapped generator, and human preference-aware scorer. Specifically, we identify and address the limitations of WebGPT (OpenAI), through which WebGLM is enabled with accuracy, efficiency, and cost-effectiveness advantages. In addition, we propose systematic criteria for evaluating web-enhanced QA systems. We conduct multi-dimensional human evaluation and quantitative ablation studies, which suggest the outperformance of the proposed WebGLM designs over existing systems. WebGLM with the 10-billion-parameter GLM (10B) is shown to perform better than the similar-sized WebGPT (13B) and even comparably to WebGPT (175B) in human evaluation.
1
4
925
Proud advisor moment: my PhD advisee, Yuxiao Dong, received the SIGKDD Rising Star Award @kdd_news. It is so well deserving for Yuxiao. #proudadvisor #datascience #KDD2022 @notredame @lucy_institute
4
💜💙🤗🤗🧡♥️
Thank you for all the LIKEs!
The top 15 most-liked organizations on @huggingface
1. @StabilityAI 20k likes
2. @AIatMeta 20k
3. @runwayml 11k
4. CompVis 10k
5. @thukeg 7k
6. @BigscienceW 7k
7. @TIIuae 7k
8. @Microsoft 6.5k
9. @GoogleAI 6k
10. @OpenAI 4k
11. @BigCodeProject 4k
12. @MosaicML 4k
13. @UKPLab 3k
14. @AiEleuther 3k
15. @salesforce 3k
huggingface.co/spaces/Pulsar…
3
1,054
happy 112-th birthday!
Get excited, everyone! Tsinghua's birthday is just around the corner! Join our #Tsinghua112 anniversary celebration on the last Sunday of April by sending your #HappyBirthdayTsinghua wish or sharing a photo of you on campus to recall #MyTsinghuaStory memories.
3
1,963
Lol. As mentioned in the blog, we are also working on quantization to make it work for 3090s (and possibly one or two A100 cards)
3
Big congrats to Prof Shimin! Very well deserved!! We’ve learned so much from him!!
Professor Hu Shimin from #Tsinghua DCST was elected as Academician of Chinese Academy of Sciences, for his great contributions to Computer Graphics, Geometric Computing and Artificial Intelligence! He also developed the widely-used DL framework Jittor. Congrats to Prof Hu! 🎉🎉🎉
3
957
Love to hear your thoughts on ChatGLM-6B and ChatGLM.cn @osanseviero @ChenhaoTan @_akhaliq @percyliang #LLM
ChatGLM-6B & ChatGLM!
ChatGLM-6B is an open CN&EN model w/ 6.2B paras (optimized for Chinese QA & dialogue for now). Trained for 1T tokens, SFT, Feedback Bootstrap, & RLHF.
w INT4 quantization, we can deploy on one 2080Ti card (6GB GPU mem required).
github.com/THUDM/ChatGLM-6B/…
2
768
Happy to see this: "RL fine-tuning with reward function from human feedback (ImageReward: arxiv.org/abs/2304.05977) can reduce bias in the pre-trained model."
#ImageReward: github.com/THUDM/ImageReward
❓ What is an effective approach for fine-tuning pre-trained t2i diffusion models using a reward function?
💡 I'm excited to share "DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models" co-led by @yingfan_bot
Website: sites.google.com/view/dpok-t…
🧵 1/N
2
897
Good point! We have scripts to perform all evaluation tasks. Let us also make the script for model training open on GitHub as well. We'll keep everyone updated on this! Thanks for the suggestion!!
1
3
Big congrats to everyone! It was led by Mr. Pengcheng Wang (the left), a master student from our KEG lab! It is about using LLMs to power digital human interactions!
Our digital humans was also used in the Beijing 2022 Winter Olympics & Paralympics games w/ sign language!
🎉🎉Hats off to the talented student teams from Tsinghua for bagging top honors at 18th "Challenge Cup". "Interactive Live Digital Human" creates interactive live digital entities with personality, powered by Large Language Models. The project gained 30K+ followers on Bilibili👏
2
1,085
Here is the link for training GLM-130B. plz feel free to let us know of any questions or issues. github.com/THUDM/LargeScale/…
1
Good point! We'd love to have it run on TPU as well but unfortunately we can't get access to TPU for testing for now ...
1
1
GLM-130B attempts to do this w/ completely open code, data, and all issues faced and lessons learned. So it could be a/the possible answer? github.com/THUDM/GLM-130B
What is the largest fully reproducible language model? That is, where I can get the data and code and run a sequence of commands that deterministically produces the exact model?
1
I'm moderating a plenary panel at #NeurIPS2023 entitled "LLMs: Beyond Scaling" with some amazing researchers.
Please send or upvote any interesting questions:
dory.app/events/2KZxWFPULUn9…
1
227
Lol 😉 Here is the ACL'22 paper on "why" GLM aclanthology.org/2022.acl-lo…
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
1