Python3 连接到 Kerberos Hbase 节俭 HTTPS 客户端python包

如何解决Python3 连接到 Kerberos Hbase 节俭 HTTPS 客户端python包

我们有 Python3 应用程序可以连接到 Hbase 并获取数据。

连接在 Kerberos Hbase Thrift 二进制协议（在 TSocket 中）运行良好，直到 Hadoop 团队将 Hadoop 系统移至 Cloudera 和 Cloudera 管理器，后者以 HTTPS 模式启动 Kerberos Hbase Thrift。

现在协议从 TSocket 更改为 HTTP/HTPS，Python 代码无法使用带有 SASL kerberos 的 HTTP 客户端进行身份验证。

Python 3.6.8 中使用的当前 Python 版本

和包版本是

节俭=0.13.0

hbase-thrift=0.20.4

pure_sasl=0.5.1

TSocket 模式下的工作代码：

############

from thrift.transport import TSocket,TTransport
from thrift.protocol import TBinaryProtocol
from hbase import Hbase
from hbase.ttypes import *
import jprops
from subprocess import call,check_output

#read cluster.properties
with open('/data/properties/cluster.properties') as fp:
properties = jprops.load_properties(fp)


# kerberos ticket
kerberos_ticket():
principal = properties["principal"]
kinitCommand = "kinit" + " " + "-kt"+ " " + keyTab + " " + principal
call(kinitCommand,shell="True")
return

# Hbase connection
def hbase_connection():
#get hbase data
thriftHost = properties["thriftHost"]
hbaseService = properties["hbaseService"]
Tsock = TSocket.TSocket(thriftHost,9090)
Tsock.setTimeout(2000000) #Milliseconds timeout
transport = TTransport.TSaslClientTransport(
Tsock,host=thriftHost,service=hbaseService,mechanism='GSSAPI'
)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = Hbase.Client(protocol)
return client,transport

#get kerberized ticket
kerberos_ticket()

client,transport = hbase_connection()
transport.open()

print(client.getTableNames())

###########

我发现在 TTransport.py 代码中有一个注释它只支持 TSocket

https://github.com/apache/thrift/blob/master/lib/py/src/transport/TTransport.py

TTransport.TSaslClientTransport

“传输：要使用的底层传输，通常只是一个 TSocket”

我们尝试使用

https://github.com/apache/thrift/blob/master/lib/py/src/transport/THttpClient.py

THttpClient.THttpClient(url) 但它不能用于 SASL kerberos 的 TTransport.TSaslClientTransport。

请帮助建议是否不能在 Cloudera 托管的 Kerberos Hbase 节俭 HTTPS 中使用 Python 以及使用 Python 连接 Hbase (Kerberos) 的任何替代方法。

PS：我通过这个链接遇到了类似的问题，但没有具体的解决方案

Python program to connect to HBase via thrift server in Http mode

提前致谢，

曼吉尔

解决方法

我找到了在 hbase http 模式下使用 kerberos 的解决方案。

客户端python包

• 六个 1.15.0
• 节俭 0.13.0
• hbase-thrift 0.20.4
• pykerberos 1.2.1

### Python code
# Prerequsite kinit and kerberos ticket is available for the user
# Hbase thrift running in http protocol secure mode
# Python code to use local kerberos ticket local cache 
# add kerberos context in http header 
# perform hbase client operation like get table,table scan etc
# 
# Important: the httpClient transport opened session will be available only for one time call,#            for next hbs operation need get new kerberos context (krb_context) by adding header and open session
##


import kerberos
from thrift import Thrift
from thrift.transport import THttpClient
from thrift.protocol import TBinaryProtocol
from hbase.Hbase import Client
import ssl


def kerberos_auth():
    hbaseService="<hbase>/<HOST>@<DOMAIN.COM>"
    #service can hbase ot HTTP based on hbase thrift configuration
    clientPrincipal="<user>@<DOMAIN.COM>"
    __,krb_context = kerberos.authGSSClientInit(hbaseService,principal=clientPrincipal)
    kerberos.authGSSClientStep(krb_context,"")
    negotiate_details = kerberos.authGSSClientResponse(krb_context)
    headers = {'Authorization': 'Negotiate ' + negotiate_details,'Content-Type':'application/binary'}
    return headers

httpClient =  THttpClient.THttpClient('https://<THRIFT_HOST>:9090/',cert_file='<client cert file path>.crt',key_file='<client cert key file path>.key',ssl_context=ssl._create_unverified_context())
# if no ssl verification is required 
# for new session start
httpClient.setCustomHeaders(headers=kerberos_auth())
protocol = TBinaryProtocol.TBinaryProtocol(httpClient)
httpClient.open()
client = Client(protocol)
# for new session end
client.getTableNames()

谢谢，
曼吉尔