網(wǎng)站首頁(yè) 編程語(yǔ)言 正文
python連接clickhouse數(shù)據(jù)庫(kù)的兩種方式小結(jié)_python
作者:挽手等風(fēng)起 ? 更新時(shí)間: 2022-07-09 編程語(yǔ)言python連接clickhouse數(shù)據(jù)庫(kù)
在Python中獲取系統(tǒng)信息的一個(gè)好辦法是使用psutil這個(gè)第三方模塊。
顧名思義,psutil = process and system utilities,它不僅可以通過(guò)一兩行代碼實(shí)現(xiàn)系統(tǒng)監(jiān)控,還可以跨平臺(tái)使用。
主要針對(duì)clickhouse_driver的使用進(jìn)行簡(jiǎn)要介紹
第一步:
- 通過(guò)pip install clickhouse_driver 安裝 clickhouse_driver
第二步:
- 方法一:使用clickhouse_driver 包中的Client類,通過(guò)實(shí)例化一個(gè)客戶端進(jìn)行對(duì)數(shù)據(jù)庫(kù)的增刪改查操作
from clickhouse_driver import Client
from datetime import datetime
import psutil
host_name = '192.168.50.94'
client = Client(host=host_name,database='default',user='default',password='自己設(shè)的密碼',send_receive_timeout=20,port=55666)
now = datetime.now()
time_stamp = now.strftime('%a %b %d %H:%M:%S CST %Y')# Tue Apr 06 15:32:55 CST 2021 <class 'str'>
create_at = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
disk_io = psutil.disk_io_counters()
net_io = psutil.net_io_counters()
chart_name = ["磁盤IO","網(wǎng)絡(luò)IO"]
metric_name1 = ["讀(數(shù)量)","寫(數(shù)量)", "讀(字節(jié))", "寫(字節(jié))", "讀(時(shí)間)", "寫(時(shí)間)"]
metric_name2 = ["發(fā)送字節(jié)數(shù)","接收字節(jié)數(shù)","發(fā)送包數(shù)","接收包"]
metric_value1 = [disk_io.read_count,disk_io.write_count,disk_io.read_bytes,disk_io.write_bytes,disk_io.read_time,disk_io.write_time]
metric_value2 = [net_io.bytes_sent,net_io.bytes_recv,net_io.packets_sent,net_io.packets_recv]
try:
for i in chart_name:
if i is "磁盤IO":
for j in metric_name1:
sql = "insert into clickhouse_host_metrics777(time_stamp,host_name, chart_name, metric_name,metric_value,create_at) " \
"values('%s','%s','%s','%s','%s','%s')" % \
(time_stamp, host_name, i, j, metric_value1[metric_name1.index(j)], create_at)
res = client.execute(sql)
elif i is "網(wǎng)絡(luò)IO":
for j in metric_name2:
sql = "insert into clickhouse_host_metrics777(time_stamp,host_name, chart_name, metric_name,metric_value,create_at) " \
"values('%s','%s','%s','%s','%s','%s')" % \
(time_stamp, host_name, i, j, metric_value2[metric_name2.index(j)], create_at)
res = client.execute(sql)
print("成功寫入數(shù)據(jù)")
except Exception as e:
print(str(e))
- 方法二:使用clickhouse_driver 包中的connect函數(shù),通過(guò)實(shí)例化一個(gè)客戶端進(jìn)行對(duì)數(shù)據(jù)庫(kù)的增刪改查操作
from datetime import datetime
import psutil
from clickhouse_driver import connect
host_name = '192.168.50.94'
#賬號(hào):密碼@主機(jī)名:端口號(hào)/數(shù)據(jù)庫(kù)
conn = connect('clickhouse://default:自己設(shè)的密碼@'+host_name+':55666/default')
cursor = conn.cursor()
now = datetime.now()
time_stamp = now.strftime('%a %b %d %H:%M:%S CST %Y')# Tue Apr 06 15:32:55 CST 2021 <class 'str'>
create_at = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
disk_io = psutil.disk_io_counters()
net_io = psutil.net_io_counters()
chart_name = ["磁盤IO","網(wǎng)絡(luò)IO"]
metric_name1 = ["讀(數(shù)量)","寫(數(shù)量)", "讀(字節(jié))", "寫(字節(jié))", "讀(時(shí)間)", "寫(時(shí)間)"]
metric_name2 = ["發(fā)送字節(jié)數(shù)","接收字節(jié)數(shù)","發(fā)送包數(shù)","接收包"]
metric_value1 = [disk_io.read_count,disk_io.write_count,disk_io.read_bytes,disk_io.write_bytes,disk_io.read_time,disk_io.write_time]
metric_value2 = [net_io.bytes_sent,net_io.bytes_recv,net_io.packets_sent,net_io.packets_recv]
try:
for i in chart_name:
if i is "磁盤IO":
for j in metric_name1:
sql = "insert into clickhouse_host_metrics777(time_stamp,host_name, chart_name, metric_name,metric_value,create_at) values('%s','%s','%s','%s','%s','%s')" % \
(time_stamp, host_name, i, j, metric_value1[metric_name1.index(j)], create_at)
# res = client.execute(sql)
res = cursor.execute(sql)
elif i is "網(wǎng)絡(luò)IO":
for j in metric_name2:
sql = "insert into clickhouse_host_metrics777(time_stamp,host_name, chart_name, metric_name,metric_value,create_at) values('%s','%s','%s','%s','%s','%s')" % \
(time_stamp, host_name, i, j, metric_value2[metric_name2.index(j)], create_at)
res = cursor.execute(sql)
cursor.close()
print("成功寫入數(shù)據(jù)")
except Exception as e:
print(str(e))
python將數(shù)據(jù)寫入clickhouse
from clickhouse_driver import Client
# connect ClickHouse
client = Client(host= ,port= ,user= ,database= , password=)
# 得到table1中查詢的數(shù)據(jù)導(dǎo)入table2中(database2中應(yīng)該事先建立對(duì)應(yīng)的table2表)
query_ck_sql = """ SELECT *
FROM database1.table1
WHERE date = today() """
# 導(dǎo)入數(shù)據(jù)到臨時(shí)表
try:
# 導(dǎo)入數(shù)據(jù)
client.execute("insert into {official_table_db}.{official_all_table_name} \
{query_ck_sql}".format(
official_table_db = database2,
official_table_name = table2,
query_ck_sql = query_ck_sql)
,types_check = True)
except Exception as e:
print str(e)
原文鏈接:https://blog.csdn.net/qq_36697196/article/details/115479096
相關(guān)推薦
- 2022-04-20 iOS實(shí)現(xiàn)多控制器切換效果_IOS
- 2023-07-09 windows找不到gpedit.msc,請(qǐng)確定文件名是否正確
- 2023-05-23 pytorch中backward()方法如何自動(dòng)求梯度_python
- 2022-07-22 Math.ceil() 函數(shù)使用介紹
- 2022-12-30 Golang反射獲取變量類型和值的方法詳解_Golang
- 2022-04-23 排序會(huì)了遞歸,不學(xué)非遞歸太可惜了
- 2023-03-22 Linux?rm命令詳解?Linux刪除文件目錄的操作方法_linux shell
- 2022-10-08 react-redux的基本使用_React
- 最近更新
-
- window11 系統(tǒng)安裝 yarn
- 超詳細(xì)win安裝深度學(xué)習(xí)環(huán)境2025年最新版(
- Linux 中運(yùn)行的top命令 怎么退出?
- MySQL 中decimal 的用法? 存儲(chǔ)小
- get 、set 、toString 方法的使
- @Resource和 @Autowired注解
- Java基礎(chǔ)操作-- 運(yùn)算符,流程控制 Flo
- 1. Int 和Integer 的區(qū)別,Jav
- spring @retryable不生效的一種
- Spring Security之認(rèn)證信息的處理
- Spring Security之認(rèn)證過(guò)濾器
- Spring Security概述快速入門
- Spring Security之配置體系
- 【SpringBoot】SpringCache
- Spring Security之基于方法配置權(quán)
- redisson分布式鎖中waittime的設(shè)
- maven:解決release錯(cuò)誤:Artif
- restTemplate使用總結(jié)
- Spring Security之安全異常處理
- MybatisPlus優(yōu)雅實(shí)現(xiàn)加密?
- Spring ioc容器與Bean的生命周期。
- 【探索SpringCloud】服務(wù)發(fā)現(xiàn)-Nac
- Spring Security之基于HttpR
- Redis 底層數(shù)據(jù)結(jié)構(gòu)-簡(jiǎn)單動(dòng)態(tài)字符串(SD
- arthas操作spring被代理目標(biāo)對(duì)象命令
- Spring中的單例模式應(yīng)用詳解
- 聊聊消息隊(duì)列,發(fā)送消息的4種方式
- bootspring第三方資源配置管理
- GIT同步修改后的遠(yuǎn)程分支