3
'jm @ s^ d dl mZ d dlmZmZmZ d dlmZmZmZ d dl m
Z
m Z m Z G dd de
e Z
dS ) )Optional)datetime timedeltatimezone)JSONDecodeErrordumpsloads)PluginIndependentPlugin PluginOptc
@ s e Zd ZdZdZdZd&ZdZdZd'Z e
ddd d
e
d dd d
e
d
ddd
e
dde dde
dde ddgZ e
e e
e dddZe
e e
e dddZdd Zd d Zd d Zd!d" Zd#d$ Zd%S )(Lokia
Collects logs and configuration from Loki.
This plugin interacts with the Loki API to fetch logs based on specified
labels and provides options for pagination and label detection.
It also collects relevant configuration files and masks sensitive
information. It works with both charmed and non-charmed Loki.
To fetch internal Loki logs, run it from the Loki container.
You can also run it from another machine and fetch only logs from
Loki API, by providing the following parameters:
`-k loki.collect-logs=true -k loki.endpoint=LOKI_URL`
Usage:
sos report -o loki -k loki.collect-logs=true -k loki.labels=severity:charm -k loki.detect-labels=true -k loki.paginate=true -k loki.endpoint=LOKI_URL
z Loki servicelokiservicesi d z collect-logsFzcollect logs from Loki API)defaultdescz
detect-labelszTfetch logs for all available labels. May result in multiple files with the same logspaginatez'fetch all available logs from Loki API.labels z1colon-delimited list of labels to fetch logs from)r Zval_typer endpointzhttp://localhost:3100zDloki endpoint to fetch logs from. Defaults to http://localhost:3100.)startendc C s` |st jtj}|s"|tdd }|jd}|jd}d| d| d| d| dtj d }|S )
N )Zdaysz%Y-%m-%dT%H:%M:%S.%fZz curl -G -s 'z3/loki/api/v1/query_range' --data-urlencode 'query={z!=~".+"}' --data-urlencode 'start=z' --data-urlencode 'end=z' --data-urlencode 'limit=z' )r nowr Zutcr Zstrftimer LOKI_QUERY_LIMIT)selfr labelr r Zstart_formattedZ
end_formattedZcommand r /usr/lib/python3.6/loki.py
query_command< s
&zLoki.query_commandc C sR | j | j||||}y t|d S tk
rL | jd| d |d S X d S )Noutputz-An error was returned from Loki API on label z-. Error message stored, not querying further.)Zexec_cmdr r r Z _log_warn)r r r r r r r r r get_logsO s z
Loki.get_logsc C sZ |d d }t tj j d }x4|D ],}x&|d D ]}t |d }t||}q4W q&W |S )Ndataresulti ʚ;valuesr )intr r timestampmin)r logsZ log_streams earliest_logstreamlogr& r r r get_earliest_log_timestamp[ s
z Loki.get_earliest_log_timestampc C s | j ||d d }| j| d}|jt|dd W d Q R X t|trLd S |r| j|}ttj j
d }d}x|t j k r||k rtj
|d } | j ||d | }
| j| d| }|jt|
dd W d Q R X t|
trd S |}| j|
}|d7 }qtW d S )Nz.log )indenti ʚ;r z.log.r )r! Zcollection_filewriter
isinstancestrr, r% r r r& r MAX_PAGINATION_ITERATIONSZ
fromtimestamp) r r r r r( Zlogfiler) Zprevious_earliest_logZiterations_countZ
log_timestampZnew_logsr r r get_logs_for_labele s.
zLoki.get_logs_for_labelc C s | j d}| j| | j d}| j| | jd | jd | jdr| jdpRd}g | _| jd}|rt|tr|r| jj|jd | jd
r| j d | d }t
|d
}| jj|d d S )Nz/etc/loki/*.yamlz/etc/worker/*.yamlz/var/log/loki/*zpebble logs loki -n 10000z collect-logsr zhttp://localhost:3100r :z
detect-labelsz curl -G -s 'z/loki/api/v1/labels'r r" ) Z path_joinZ
add_copy_specZadd_cmd_output
get_optionr r0 r1 extendsplitZcollect_cmd_outputr )r Zels_config_fileZ coordinated_workers_config_filer Z
labels_optionZ
labels_cmdZ labels_jsonr r r setup s$
z
Loki.setupc C s: | j dp d}x&| jD ] }| j d}| j||| qW d S )Nr zhttp://localhost:3100r )r5 r r3 )r r r r r r r collect s
z Loki.collectc C sh ddg}dddddg}dd j | d
}dd j | d }x(|D ] }| j||d | j||d q@W d S )
NZ
access_key_idZsecret_access_keyz /etc/loki/loki-local-config.yamlz/etc/loki/config.yamlz/etc/loki/local-config.yamlz/etc/loki/loki.yamlz/etc/worker/config.yaml(|z)\s*(:|=)(\S*\n.*?\\n)z)\s*(:|=)\s*[a-zA-Z0-9]*z
\1\2*********)joinZ do_file_sub)r Z protect_keysZ
loki_filesZmatch_exp_multilZ match_expfiler r r postproc s
z
Loki.postprocN)r )r
)__name__
__module__ __qualname____doc__Z
short_descZ plugin_nameZprofilesr r2 Zpackagesr r1 Z option_listr r r r! r, r3 r8 r9 r> r r r r r s4
r N)Ztypingr r r r Zjsonr r r Zsos.report.pluginsr r
r r r r r r s