Mj- 8 d Z dZddlZddlZddlZddlZddlmZ ddlZ ddl
m Z ddl m
Z
ddlmZmZ d d
lmZ d d lmZ d d lmZ d d
lmZmZ dZ G d de
Ze dk r7 e ej dej e Z e dS dS )z*
This module contains DecisionMaker class
z ssa.modules N)isfile)setup_database )Common)iter_domains_dataiter_domain_url_data )
load_tunables)ssa_tunables_schema) report_path)previous_day_date sentry_initi c \ e Zd ZdZd fd Zd Zedefd Ze defd Z
e de fd Z e de
fd Zed
Zde fd Zd edefd
ZdedefdZdedefdZededededefd Zd Zde fdZde fdZdedefd Zd Z d ededefd Z xZ S )!
DecisionMakerz,
SSA Decision maker implementation.
Nc t t j d | _ | j dt |r|n
t | _ d S )Ndecision_makerzDecisionMaker enabled: %s) super__init__logging getLoggerloggerinfo __package__r engine)selfr __class__s Q/opt/cloudlinux/venv/lib64/python3.11/site-packages/ssa/modules/decision_maker.pyr zDecisionMaker.__init__, s_
'(899 4kBBB &z1DecisionMaker.data_processing.. s X X XAuu X X Xr T)reverser4 )r- slow_urls slow_reqs
total_reqsurlsrK list_to_sort leave_topkey_for_sorting)#r6
isinstance urls_numberintr r listr domain_name
is_ignoredr r non_url_fieldsis_throttling_suitablegetget_correlationdomain_total_reqserrorstrrequest_number_exceededcorrelation_conditionsnpmeansumr5 MAX_REPORT_URL_LENfloatlenheapqheappush heapreplacesortedappendreport_sortingdomains_number)r r& rP domain_data urls_dataorderdomain_slow_reqsurl durationsurl_datacorrelation_value average_duration_calculationsum_url_slow_reqsrG item sorted_urlss r r$ z DecisionMaker.data_processing[ s1 # )343CS(I(IidN^abNbNbD$$hi ,T[99 : : K IE ,@kNe,f,f !
!
(Y??3'' K%%lC888 $--- 22 LL!5tQC"H~~FFQaHb $($8$8BR9SU`Ur$s$s! K%%&FCQQQ // 0IJJ tOjOj%P P 46793E3E0(+H_,E(F(F%$(99$ !4"4!4 5!2),-I)J)J$)->*D*D$E$E E .vu=D9~~ 11 y$7777 ! ,, ))T:::QJE
Y XyRV8W8W8W X X X y!(( ( 4"%k"2"2"2#&{'D#E#E ( ) $ 3 3 #I .$:M_j !4 ! !F9
r rA c n | j pdfdt dt D S )z
Based on the 'url_slow_reqs' list, a new list will be formed,
where the elements of the original list will be iteratively
summed by the number of elements equal to 'time'
r? c D g | ] }t ||z S r9 )rc )rE itimerA s r rH z@DecisionMaker.list_handling_considering_time.. s. ]]]QM!dQh,/00]]]r r )r} rangerf )r rA r} s `@r list_handling_considering_timez,DecisionMaker.list_handling_considering_time sC yB]]]]]%3}CUCUW[:\:\]]]]r url_slow_reqs_by_timec , |D ]}|| j k r dS dS )z_
This functions will check if any of elements is greater than "request_number"
TF)request_number)r r r| s r $compare_elements_with_request_numberz2DecisionMaker.compare_elements_with_request_number s2 ' AD'''tt (ur r@ r\ c b | j sdS t j t j || S )z}
Calculates the correlation coefficient using the "url_total_reqs" and
the "domain_total_reqs" lists
r )r8 ra amincorrcoef)r r@ r\ s r r[ z DecisionMaker.get_correlation s1
1wr{>3DEEFFFr rO rP rQ c T | fdd |r
| d| S | S )a
Will sort the domain list by "slow_reqs", the goal is to leave only
"domains_number" of uppers, also per each domain will sort urls by
"reqs_num", the goal is to leave only "urls_number" of uppers.
leave_top == 0 allows to keep the full list
c | S Nr9 )dict_rQ s r z.DecisionMaker.report_sorting.. s E/,B r T)keyrI N)sortrN s `r rl z DecisionMaker.report_sorting sE B B B BDQQQ
+ + r c | j }t | rt | 5 } t j | }| dd dd }n# t j $ r d}Y nw xY wddd n # 1 swxY w Y d| d}| | }t j
|| dS dS ) z#
Rename old report
r3 z
dd.mm.yyyy.rF unknownNreport__z.json) r1 r openjsonloadrZ replaceJSONDecodeErrorr. r* rename)r
old_report json_dataddate_from_reportnew_report_name
new_reports r rename_old_reportz DecisionMaker.rename_old_report s@ -
*
.j!!
UYU ),,A ()uuV\'B'B'J'J3PS'T'T$$ + 1 1 1'0$$$1
U
U
U
U
U
U
U
U
U
U
U
U
U
U
U A)9 @ @ @O **?;;J Ij* - - - - -
.
.s3 B
A'+B
'A;8B
:A;;B
BBr& c | t | j dd 5 }t j ||dd ddd dS # 1 swxY w Y dS )z#
Makes json report
wzutf-8)encodingF ) ensure_asciiindentN)r r r1 r dump)r r& fs r r% z DecisionMaker.add_json_report s
$*C'
B
B
B ?a IfaeA > > > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?s AAAc d} t | j 5 }t j || }ddd n # 1 swxY w Y n## t t j f$ r
| j }Y nw xY w|S )zT
Return contents of current report or empty report in case of error
N) object_hook)r r1 r r OSErrorr r6 )r _filtering_hookr& report_dicts r get_json_reportz DecisionMaker.get_json_report s -d.//
M6 "iO L L L
M
M
M
M
M
M
M
M
M
M
M
M
M
M
M-. - - - ,KKK -s. A ; A ?A ?A A'&A'ru c * | j sdS || j k S )a
If correlation flag is enabled - we'll compare correlation_coefficient
from configuration with calculated correlation coefficient.
If the calculated value exceeds the configuration value - we return
True otherwise False. At the same time if correlation flag is disabled -
we'll also return "True" since in this case the correlation coefficient
is not checked and its value is specified as zero in final report.
T)r8 correlation_coefficient)r ru s r r` z$DecisionMaker.correlation_conditions s" 4 4#???r c V | | }| | S )z
At least one element from the received list (url_slow_reqs_by_time)
must be greater than request_number
)r r )r rA r s r r_ z%DecisionMaker.request_number_exceeded s-
!% C CM R R 889NOOOr r> c t | t | z dz }| j d| || j dd k S )zY
Check that percent of throttled requests per URL passes given threshold
d z Calculated throttled percent %s allowed_throttling_percentager )rc r r r# rZ )r r> r@ throttled_percents r rY z$DecisionMaker.is_throttling_suitable s_ !!3 4 4s>7J7J Jc Q ;=NOOO D$:$>$>?^`a$b$bbbr r ) __name__
__module__ __qualname____doc__r r' staticmethodr^ r. propertyr1 r5 r6 setr: r" r$ rU r boolr r[ rT rl r r% r r` r_ rY
__classcell__)r s @r r r ' s = = = = = = /c / / / \/ 0S 0 0 0 X0 :t : : : X: s X > > \>P P P P Pd^D ^T ^ ^ ^ ^$ SW Gd Gt G G G G T c C TX \ . . ."?d ? ? ? ? @ @ @ @ @ @P P Pc ct cX\ c c c c c c c cr r __main__z decision_maker_standalone.log)filenamelevel) r r rg r r r* os.pathr numpyra ssa.dbr commonr storager r
configurationr
configuration.schemesr internal.constantsr internal.utilsr
r rd r r basicConfigINFOdmr9 r r r s ! ! ! ! ! ! * ) ) ) ) ) 7 7 7 7 7 7 , , , , , , ; ; ; ; ; ; ; ;
pc pc pc pc pcF pc pc pcf zKMMMG!@ UUUU BBDDDDD r