3
\mee @ s d Z ddlZddlZddlZddlZddlZddlZddlZddlm Z ddl
m Z ddl m
Z
ddlmZ ddlmZ ddlmZ dd lmZmZmZmZmZmZmZm Z ddl Z dd
l m Z m Z dd l!m"Z" dd l#m$Z$m%Z% dd
l&m'Z' ddl(m)Z)m*Z*m+Z+ yddl,m-Z. e.Z-W n e/k
r. e0Z-Y nX ej1e2Z3dZ4dZ5dZ6de5ide6idZ7ddddZ8dZ9dZ:dZ;d Zd Z?d Z@eAd d! e$jB D ZCerddlZdd"lDmEZEmFZF G d#d$ d$eFZGneHZGG d%d& d&e-ZIG d'd( d(eZJeeJ ZKG d)d* d*eZLdd+d,eKeeM eMeMd-d.d/ZNG d0d1 d1eOZPd2d3 ZQdeHeMd5d6d7ZRee- d8d9d:ZSeIfeTeHeMee0 d;dZVeeUd?d>ZWeIfee0 d@dAdBZXeIfee0 d@dCdDZYdEdF ZZdeHdHdIdJZ[deHeeH eMeTeTeTeTdLdMdNZ\G dOdP dPZ]dddQeHeMeHeeK eeK eeMe^f dRdSdTZ_eMeeJ dUdVdWZ`deMeMeMeLdXdYdZZadeMeHeMeTeMeTd[d\d]Zbd^d_ Zcd`da ZdeHeMeeM dbdcddZeeHeHeMdedfdgZfeMd8dhdiZgeHeHdjdkdlZheHdmdndoZieHeMdpdqdrZjdeHeHeMdtdudvZkeGeMdwdxdyZleGeMdwdzd{ZmdeGeeH eMd|d}d~ZneHd8ddZoepeMdddZqeMd8ddZrdeMeHdddZsdddZtdd Zudd Zve2dkrejwev dS )zFschema.py: Set of module functions for processing cloud-config schema. N) defaultdict)Iterable)deepcopy)EACCES)partial)chain)
TYPE_CHECKING DefaultDictList
NamedTupleOptionalTypeUnioncast)importersafeyaml)read_cfg_paths)INCLUSION_TYPES_MAPtype_from_starts_with)DataSourceNotFoundException)errorget_modules_from_dir load_file)ValidationErrorz!versions.schema.cloud-config.jsonzschema-cloud-config-v1.jsonz schema-network-config-v1.jsonlatest)z cloud-configznetwork-configtrueZfalseZnull)TFNap
{name}
{title_underbar}
{title}
.. tab-set::
{prefix3}.. tab-item:: Summary
{description}
{prefix6}**Internal name:** ``{id}``
{prefix6}**Module frequency:** {frequency}
{prefix6}**Supported distros:** {distros}
{prefix6}{activate_by_schema_keys}
{prefix3}.. tab-item:: Config schema
{property_doc}
{prefix3}.. tab-item:: Examples
{prefix6}::
{examples}
z5{prefix}* **{prop_name}:** ({prop_type}){description}zJ{prefix}* Each object in **{prop_name}** list supports the following keys:z&
# --- Example{example_count} ---
deprecatedz DEPRECATED: c C s g | ]}|d dkr|qS )r # ).0tr r /usr/lib/python3.6/schema.py
p s r# ) NotRequired TypedDictc @ sB e Zd ZU eeeeej e
ej e e e
e e dS )
MetaSchemaN)__name__
__module__ __qualname__strnameidtitle descriptiontypingr
distrosexamples frequencyr$ activate_by_schema_keysr r r r" r& z s
r& c @ s e Zd ZdS )SchemaDeprecationErrorN)r' r( r) r r r r" r4 s r4 c @ s$ e Zd ZU eeedddZdS )
SchemaProblem)returnc C s | j d| j S )Nz: )pathmessage)selfr r r" format s zSchemaProblem.formatN)r' r( r) r* r7 r8 r: r r r r" r5 s
r5 c @ s e Zd ZU eedS ) UserDataTypeAndDecodedContentN)r' r( r) r*
userdata_typecontentr r r r" r; s
r; z, )prefix separator)schema_problemsr> r? r6 c C s( |j tdd | }|r$| | }|S )Nc S s | j S )N)r: )pr r r" s z)_format_schema_problems..)joinmap)r@ r> r? Z formattedr r r" _format_schema_problems s rE c s@ e Zd ZdZd ee ee d fdd
ZedddZ Z S )
SchemaValidationErrorz )sortedlistsetrE )problemsr> )r8 r r" handle_problems s z7SchemaValidationError.__init__..handle_problemsz Cloud config schema errors: )r> z"Cloud config schema deprecations: N)rG rH super__init__)r9 rG rH rM ) __class__)r8 r" rO s
z SchemaValidationError.__init__)r6 c C s
t | jS )N)boolrG )r9 r r r"
has_errors s z SchemaValidationError.has_errors)NN)
r' r( r) __doc__r SchemaProblemsrO rQ rR
__classcell__r r )rP r" rF s rF c C s@ yddl m} W n tk
r$ dS X |jj|dp>t|tfS )zWTYPE_CHECKER override allowing bytes for string type
For jsonschema v. 3.0.0+
r )Draft4ValidatorFstring)
jsonschemarV ImportError TYPE_CHECKERis_type
isinstancebytes)ZcheckerinstancerV r r r" is_schema_byte_string s r_ F)configr6 c sR t d fdd }|r |ndddg}djt||}jdd}| | j S ) zcombine description with new/changed/deprecated message
deprecated/changed/new keys require a _version key (this is verified
in a unittest), a _description key is optional
)keyc sr j | sdS j | dd}j | dd| d}| j d| d| } rbd| S d |j d
S ) Nr Z _descriptionZ_versionz z in version z. z
**)get
capitalizestrip)ra Zkey_descriptionvmsg)annotater` r r" format_message s
z:_add_deprecated_changed_or_new_msg..format_messager changednewr r. )r* rC rD rd rstrip)r` ri
filter_keyrj Z filter_keysZchanged_new_deprecatedr. r )ri r` r" "_add_deprecated_changed_or_new_msg s
ro )r6 c C s g }d}t |tod|k}x| D ]}|r|d |jjdi jdi jdg krT|gS t|dr||jdd dkr|j| q|jr|jd dkr|j| q t|j|kr|j| q t|j|kr t|j}|g}q W |S ) zReturn the best_match errors based on the deepest match in the json_path
This is useful for anyOf and oneOf subschemas where the most-specific error
tends to be the most appropriate.
r type
propertiesenum json_path N) r\ dictschemard hasattrrs appendr7 len)errorsr^ Z best_matchesZ
path_depthr[ errr r r" cloud_init_deepest_matches s(
r} )r rw rn
error_typec c s" |r t |d|gd}||V dS )zJsonschema validator for `deprecated` items.
It raises a instance of `error_type` if deprecated that must be handled,
otherwise the instance is consider faulty.
T)ri rn N)ro )
_validatorr Z _instancerw rn r~ rh r r r" r # s
r )rn rk )r~ c # s ddl m} g }g }d}xt|D ]\} }
t| j||
| d} tt fdd| } tt fdd| }
| sz|j|
P t|trd|krd |
j d
d krd |d |
d
krd
}| E dH |j| q"W |s||V t
d|f |dV |E dH dS )a Jsonschema validator for `anyOf`.
It treats occurrences of `error_type` as non-errors, but yield them for
external processing. Useful to process schema annotations, as `deprecated`.
Cloud-init's network schema under the `config` key has a complexity of
allowing each list dict item to declare it's type with a `type` key which
can contain the values: bond, bridge, nameserver, physical, route, vlan.
This schema 'flexibility' makes it hard for the default
jsonschema.exceptions.best_match function to find the correct schema
failure because it typically returns the failing schema error based on
the schema of greatest match depth. Since each anyOf dict matches the
same depth into the network schema path, `best_match` just returns the
first set of schema errors, which is almost always incorrect.
To find a better schema match when encountering schema validation errors,
cloud-init network schema introduced schema $defs with the prefix
`anyOf_type_`. If the object we are validating contains a 'type' key, and
one of the failing schema objects in an anyOf clause has a name of the
format anyOf_type_XXX, raise those schema errors instead of calling
best_match.
r )
best_matchF) schema_pathc s t | S )N)r\ )e)r~ r r" rB b s z_anyOf..c s
t | S )N)r\ )r )r~ r r" rB d s rp Z
anyOf_typez$refr Z anyOf_type_TNz.%r is not valid under any of the given schemas)context) jsonschema.exceptionsr enumeraterJ descendfilterextendr\ rv rd r ) validatoranyOfr^ _schemar~ r
all_errorsall_deprecationsZskip_best_matchindex subschemaall_errserrs deprecationsr )r~ r" _anyOf; s4
r c # s t |}g }g }x~|D ]f\}} tj| |d}
tt fdd|
} tt fdd|
} | sr| }
|j| P |j| qW t|E dH fdd|D }|r|j|
djd d
|D }td |f V n
|E dH dS ) zJsonschema validator for `oneOf`.
It treats occurrences of `error_type` as non-errors, but yield them for
external processing. Useful to process schema annotations, as `deprecated`.
)r c s t | S )N)r\ )r )r~ r r" rB s z_oneOf..c s
t | S )N)r\ )r )r~ r r" rB s Nc s g | ]\}}j |r|qS r )is_valid)r is)r^ r r r" r# s z_oneOf..z, c s s | ]}t |V qd S )N)repr)r rw r r r" s z_oneOf..z %r is valid under each of %s) r rJ r r r r} ry rC r )r oneOfr^ r r~ Z
subschemasr r r r r r r Z first_validZ
more_validZreprsr )r~ r^ r r" _oneOf~ s,
r c
C s ddl m} m} ddlm} t| j}ddi|d d< i }t| drdd |d
< | jj dt
}d |i}n| j }t t
f|d< d |i}t| j}t|t< t|d
< t|d< t|d< |f ||dd|}ddd} | |_||fS )zGet metaschema validator and format checker
Older versions of jsonschema require some compatibility changes.
@returns: Tuple: (jsonschema.Validator, FormatChecker)
@raises: ImportError when jsonschema is not present
r )rV
FormatChecker)createrp rW rq labelrZ FadditionalProperties type_checkerZ
default_typesrk r r Zdraft4) meta_schema
validatorsversionNc [ s$ t dd | j||}t|ddkS )zgOverride version of `is_valid`.
It does ignore instances of `SchemaDeprecationError`.
c S s t | t S )N)r\ r4 )r r r r" rB s z.is_valid..N)r iter_errorsnext)r9 r^ r __r{ r r r" r s z*get_jsonschema_validator..is_valid)N)rX rV r Zjsonschema.validatorsr r Z META_SCHEMArx rZ Zredefiner_ Z
DEFAULT_TYPESr* r] rv Z
VALIDATORS_validator_deprecatedDEPRECATED_KEY_validator_changedr r r )
rV r r r Zvalidator_kwargsr typesr cloudinitValidatorr r r r" get_jsonschema_validator s4
r T)rw c C sz ddl m} y| j| W nZ |k
rt } z>|rXttdjdd |jD |jgd|tj d| W Y dd}~X nX dS ) a Validate provided schema meets the metaschema definition. Return strict
Validator and FormatChecker for use in validation
@param validator: Draft4Validator instance used to validate the schema
@param schema: schema to validate
@param throw: Sometimes the validator and checker are required, even if
the schema is invalid. Toggle for whether to raise
SchemaValidationError or log warnings.
@raises: ImportError when jsonschema is not present
@raises: SchemaValidationError when the schema is invalid
r ) SchemaError.c S s g | ] }t |qS r )r* )r rA r r r" r# s z3validate_cloudconfig_metaschema..)rG zGMeta-schema validation failed, attempting to validate config anyway: %sN)
r r Z check_schemarF r5 rC r7 r8 LOGwarning)r rw throwr r| r r r" validate_cloudconfig_metaschema s
r cloud-config)r` rw schema_typestrictstrict_metaschema log_detailslog_deprecationsc
C sx |dkrt |}y t \}}|r.t||dd W n tk
rN tjd dS X ||| d} g }
g } xt| j| dd dD ]x} d jd
d | j D }
|
r| j
d kr| j |krt j
d
| j}|r|d }
t|
| jf}t| tr| |7 } q~|
|7 }
q~W |r | r t| ddd}tj| |r:|
s0| r:t|
| |
rt|r^t|
d| ddd}n d| d}tj| dS )a Validate provided config meets the schema definition.
@param config: Dict of cloud configuration settings validated against
schema. Ignored if strict_metaschema=True
@param schema: jsonschema dict describing the supported schema definition
for the cloud config module (config.cc_*). If None, validate against
global schema.
@param schema_type: Optional string. One of: cloud-config, network-config
Default: cloud-config.
@param strict: Boolean, when True raise SchemaValidationErrors instead of
logging warnings.
@param strict_metaschema: Boolean, when True validates schema using strict
metaschema definition at runtime (currently unused)
@param log_details: Boolean, when True logs details of validation errors.
If there are concerns about logging sensitive userdata, this should
be set to False.
@param log_deprecations: Controls whether to log deprecations or not.
@raises: SchemaValidationError when provided config does not validate
against the provided schema.
@raises: RuntimeError when provided config sourced from YAML is not a dict.
@raises: ValueError on invalid schema_type not in cloud-config or
network_config
NF)r z5Ignoring schema validation. jsonschema is not present)Zformat_checkerc S s | j S )N)r7 )r r r r" rB I s z-validate_cloudconfig_schema..)ra r c S s g | ] }t |qS r )r* )r rA r r r" r# K s z/validate_cloudconfig_schema..r z#.*\('(?P.*)' was unexpected\)r+ z"Deprecated cloud-config provided:
)r> r? zInvalid z provided:
zQ provided: Please run 'sudo cloud-init schema --system' to see the schema errors.T)
get_schemar r rY r debugrI r rC r7 r rw rematchr8 r5 r\ r4 rE r rF r )r` rw r r r r r r r r r{ r schema_errorr7 Z
prop_matchproblemr8 Zdetailsr r r" validate_cloudconfig_schema sZ !
r c @ s e Zd ZeeedddZeeee edddZe ddd Z
edee ee ee e ee d d d
Z ee eeee dddZ
e e edddZdS )
_Annotator) cloudconfigoriginal_content schemamarksc C s || _ || _|| _d S )N) _cloudconfig_original_content _schemamarks)r9 r r r r r r" rO | s z_Annotator.__init__)r- r= r6 c C s dj |}d| d| dS )Nr z# z: -------------
z
)rC )r- r= Zbodyr r r"
_build_footer s
z_Annotator._build_footer)r@ c C s~ t t}xp|D ]h\}}tjd|}|rF|j \}}|t| j| nd }|| j| j| |d k rdj|||d}qW |S )Nz&format-l(?P\d+)\.c(?P\d+).*z Line {line} column {col}: {msg})linecolrh ) r rJ r r groupsintry r r: )r9 r@ errors_by_liner7 rh r r r r r r" _build_errors_by_line s z _Annotator._build_errors_by_liner )rL labelsfooterr label_prefixr6 c C sF x@| D ]8}| | }|j | |j d| d| |d7 }qW |S )Nz# z: )ry )rL r r r r r r r r r"
_add_problems s
z_Annotator._add_problems)linesr deprecations_by_liner6 c s g }g }g }d}d}x~t |dD ]p\} }
|| } || } | s@| rg }
j| |
||dd} j| |
||dd}|j|
d dj|
q |j|
q W |jt fddtd d d
|fd |ff |S ) Nr E)r Dz # ,c s
j | S )N)r )seq)r9 r r" rB s z._Annotator._annotate_content..c S s t | d S )Nr )rQ )r r r r" rB s ErrorsZ Deprecations)r r ry rC r rD r )r9 r r r annotated_contentZ error_footerZdeprecation_footerZ error_indexZdeprecation_indexZ line_numberr r{ r r r )r9 r" _annotate_content s8
z _Annotator._annotate_content)rG rH r6 c C sp | r| r| j S | j jd}t| jtsDdj|| jddgg S | j|}| j|}| j|||}dj|S )Nr r z&# E1: Cloud-config is not a YAML dict.) r splitr\ r rv rC r r r )r9 rG rH r r r r r r r" ri s
z_Annotator.annotateN)r )r' r( r) rv r* rO staticmethodr
r rT r r r r ri r r r r" r { s
)r )rG rH )r r r rG rH r6 c C s t | ||j|pg |pg S )a Return contents of the cloud-config file annotated with schema errors.
@param cloudconfig: YAML-loaded dict from the original_content or empty
dict if unparseable.
@param original_content: The contents of a cloud-config file
@param schemamarks: Dict with schema marks.
@param schema_errors: Instance of `SchemaProblems`.
@param schema_deprecations: Instance of `SchemaProblems`.
@return Annotated schema
)r ri )r r r rG rH r r r" annotated_cloudconfig_file s r )r= r6 c C sf ddl m} || krg S g }xDt| j dD ]2\}}|j|r,|jtd| d|j|d q,W |S )a Annotate and return schema validation errors in merged cloud-config.txt
When merging multiple cloud-config parts cloud-init logs an error and
ignores any user-data parts which are declared as #cloud-config but
cannot be processed. the hanlder.cloud_config module also leaves comments
in the final merged config for every invalid part file which begin with
MERGED_CONFIG_SCHEMA_ERROR_PREFIX to aid in triage.
r ) MERGED_PART_SCHEMA_ERROR_PREFIXr zformat-lz.c1zIgnored invalid user-data: )Z cloudinit.handlers.cloud_configr r
splitlines
startswithry r5 replace)r= r r{ Zline_numr r r r" )process_merged_cloud_config_part_problems s
r ) config_pathr= instance_data_pathr6 c C s ddl m}m}m} t|}d}|dkry||| |}W nd |k
rl } ztt|dg|W Y dd}~X n2 |k
r } ztt|dd W Y dd}~X nX d }t|}|s|j d
\} }
}
tt|d | d | d
dj
t gn|dkrt d| d t
||S )a
Return tuple of user-data-type and rendered content.
When encountering jinja user-data, render said content.
:return: UserDataTypeAndDecodedContent
:raises: SchemaValidationError when non-jinja content found but
header declared ## template: jinja.
r )JinjaLoadError
NotJinjaError render_jinja_payload_from_filez format-l1.c1z text/jinja2zRDetected type '{user_data_type}' from header. But, content is not a jinja templateNT)sys_exitz format-l2.c1r z!Unrecognized user-data header in z: "z%".
Expected first line to be one of: z, ztext/cloud-configzUser-data type 'z.' not currently evaluated by cloud-init schema)Z!cloudinit.handlers.jinja_templater r r r rF r5 r r* partitionrC USERDATA_VALID_HEADERSprintr; ) r r= r r r r Zuser_data_typeZschema_positionr Z header_line_r r r" &_get_config_type_and_rendered_userdata& s6 "$
r )r rw r ri r r6 c " C s t | dd}|s$td|| f dS |d kr8t||}n t| ||}|jd krRdS |j}t|}y&|rvtj|\} }
ntj |} i }
W n t
j k
r` } zd } }
d}t | d rt
| d rt
| d }n t | d
rt
| d
rt
| d
}|r |jd } |jd }
|jtd j| |
d d
j| t| t|}|rJtti |i |jd || W Y dd} ~ X nX t| ts|st| d| d|dkr| jd| }|std dS |jddkrtd|jd dS y*t| |ddds td| d dS W n tk
r } zl| j r*|| j7 }|rJtt| ||
|| j d n | j rjt | j ddd}t| |r|t|d| W Y dd} ~ X nX dS ) a Validate cloudconfig file adheres to a specific jsonschema.
@param config_path: Path to the yaml cloud-config file to parse, or None
to default to system userdata from Paths object.
@param schema: Dict describing a valid jsonschema to validate against.
@param schema_type: One of network-config or cloud-config.
@param annotate: Boolean set True to print original config file with error
annotations on the offending lines.
@param instance_data_path: Path to instance_data JSON, used for text/jinja
rendering.
:return: True when validation was performed successfully
:raises SchemaValidationError containing any of schema_errors encountered.
:raises RuntimeError when config_path does not exist.
T)decodez,Empty '%s' found at %s. Nothing to validate.Fnetwork-configtext/cloud-configr NZ context_markZ problem_markzformat-l{line}.c{col})r r z File {0} is not valid YAML. {1})rG rb z is not a YAML dict.Znetworkz:Skipping network-config schema validation on empty config.r zJSkipping network-config schema validation. No network schema for version: )r r z Skipping z2 schema validation. Jsonschema dependency missing.)rG rH z"Cloud config schema deprecations: z, )r> r? )r )r r ) r r r; r r< r= r r Zload_with_marksloadyamlZ YAMLErrorrx getattrr columnry r5 r: r* rF r rG r\ rv RuntimeErrorrd r rR rH rE )r rw r ri r Zdecoded_contentZdecoded_configr= r{ r Zmarksr r r Zmarkr Znetcfgr8 r r r" validate_cloudconfig_filec s
r c C s | dkr dS | dkrdS dS )zProvide a sorting weight for documentation of property types.
Weight values ensure 'array' sorted after 'object' which is sorted
after anything else which remains unsorted.
Zarray objectr r r )valuer r r" _sort_property_order s
r c c sB x<| D ]4}t |tr4t |ttf r4t|E d H q|V qW d S )N)r\ r r* r] _flatten)Zxsxr r r" r s
r )
property_dict multi_keyr6 c C sr g }x`| j |i D ]P}|j tr"q|j drJ|jdd |j dg D q|j dr|j|d qW tt|S )Nrr c S s g | ]}d t j|| d qS )z``) _YAML_MAPrd )r Z
enum_valuer r r" r# s z,_collect_subschema_types..rp )rd r r ry rJ r )r r property_typesr r r r" _collect_subschema_types s
r )r defsr6 c
C s| t | | | jdg }t|ts&|g}| jdrDdd | d D }n6| jdr`|jt| d n| jdrz|jt| d t|dkr|d }n|jtd d
j |}| jd i }|jdg }t|ts|g}t
|}xHt |jdi |jdi D ]*}t ||}|r|d krq|j
| qW |rrt|dkrF| d
|d S |jtd dd
j | d} | d
| S |pzd S )zNReturn a string representing a property type from a given
jsonschema.
rp rr c S s g | ]}d t j|| d qS )z``)r rd )r kr r r" r# s z&_get_property_type..r r r r )ra /itemsZ UNDEFINEDz of ())_flatten_schema_refsrd r\ rJ r r rz sortr rC rQ r _get_property_typery )
r r r Z
property_typer Zsub_property_typesZprune_undefinedsub_itemZsub_typeZsub_property_docr r r" r s>
r c C s> |}t jdd| } t jddj|| } t jddj|| } | S )a Parse description from the meta in a format that we can better
display in our docs. This parser does three things:
- Guarantee that a paragraph will be in a single line
- Guarantee that each new paragraph will be aligned with
the first paragraph
- Proper align lists of items
@param description: The original description in the meta.
@param prefix: The number of spaces used to align the current description
z
(\S)\n(\S)z\1 \2z\n\nz\n\n{}z\n( +)-z\n{}-)r subr: )r. r> Zlist_paragraphr r r" _parse_description5 s r )src_cfgr c C s d| kr(| j djdd}| j|| d| krd| d krd| d j djdd}| d j|| d| d krx<| d d D ],}d|kr~|j djdd}|j|| q~W xTt| jdg | jdg | jdg D ],}d|kr|j djdd}|j|| qW dS ) zEFlatten schema: replace $refs in src_cfg with definitions from $defs.z$refz#/$defs/r r r r allOfN)popr updater rd )r r Z reference
sub_schemar r r" r K s&
r )r c C s0 | j dd}|sdS x|D ]}| j| qW dS )zFlatten schema: Merge allOf.
If a schema as allOf, then all of the sub-schemas must hold. Therefore
it is safe to merge them.
r N)r r )r Z sub_schemasr r r r" _flatten_schema_all_ofe s
r ) prop_configr6 c C s t ttddd}| jdi }| jdi }g }g }|| || x t||D ]}|||| qJW djt||}|r|d| }|S )aG Return accumulated property description.
Account for the following keys:
- top-level description key
- any description key present in each subitem under anyOf or allOf
Order and deprecated property description after active descriptions.
Add a trailing stop "." to any description not ending with ":".
)r` descriptionsdeprecated_descriptionsc S s< t t| jdr |jt| n| jdr8|jt| d S )Ndeprecated_versionchanged_version new_versionr. )r r r )anyrD rd ry ro )r` r r r r r" assign_descriptions} s
z6_get_property_description..assign_descriptionsr r z. rb )rv rJ rd r rC )r r r r r r r r. r r r" _get_property_descriptionr s
r )rw r r6 c
s |d }g } j ddkr dS fdddD } fd d|D }x\|D ]R}xJ|j D ]<\}} t| | t| | j ddkrq\t| }
| j d
|} |jtj|| t|
|d t | |d | j d } | rtt| | | j ds| j dr |jt
j|| d
|jt | ||d xV| j dg D ]F}
|
j dsF|
j dr*|jt
j|| d
|jt |
||d q*W d| ksd| kr\|jt | ||d q\W qJW dj |S )zDReturn restructured text describing the supported schema properties.z hiddenTr c s$ g | ] }d ks | d kr|qS )r r )r ra )rw r r" r# s z%_get_property_doc..rq patternPropertiesc s g | ]} j |i qS r )rd )r ra )rw r r" r# s r )r> prop_namer. Z prop_typer )r> r! )r r> r z
)rq r )
rd r r r r ry SCHEMA_PROPERTY_TMPLr: r r SCHEMA_LIST_ITEM_TMPL_get_property_docrC )rw r r> Z
new_prefixrq Z
property_keysZproperty_schemasZ prop_schemaZprop_keyr r. r r Z
alt_schemar )rw r" r$ sZ
r$ )metar6 c C sb | j d}|sdS t}xFt|dD ]8\}}|tj|d7 }tj|djd}|dj|7 }q"W |S )zAReturn restructured text describing the meta examples if present.r1 r r )Z
example_countz r ) rd SCHEMA_EXAMPLES_HEADERr SCHEMA_EXAMPLES_SPACER_TEMPLATEr: textwrapindentr rC )r% r1 Z rst_contentcountZexampleZindented_linesr r r"
_get_examples s
r+ c C s2 | j dsdS djdd | d D }d| dS )Nr3 r z, c s s | ]}d | d V qdS )z``Nr )r r r r r" r s z3_get_activate_by_schema_keys_doc..z**Activate only on keys:** z
)rd rC )r% Z schema_keysr r r" _get_activate_by_schema_keys_doc s
r, )r% rw r6 c
C s |dkrt dd}| s | r&tdt| j }ddddd d
d h}d h}d
}|| rfdj|| }n|| | rdj|| }|rt|tt| }d
|d< d|d< d|d< tj t
t |d
d|d
< |j di }|j | d r|j | d i }t
t|}t
|d j rPyt||dd|d< W n( tk
rN tjd d
|d< Y nX |j dd
sfd|d< tj t| d|d< |d sd|d< d j| d |d < tjd d | d |d < t| |d < tjf |}|S ) zReturn reStructured text rendering the provided metadata.
@param meta: Dict of metadata to render.
@param schema: Optional module schema, if absent, read global schema.
@raise KeyError: If metadata lacks an expected key.
Nz cloud-config)r z"Expected non-empty meta and schemar, r- r1 r2 r0 r. r+ r3 r z(Missing required keys in module meta: {}z3Additional unexpected keys found in module meta: {}Zproperty_headerz Zprefix6z Zprefix3z$defsrq )r r> Z property_docz3Unable to render property_doc due to invalid schemaz+ No schema definitions for this modulez$ No examples for this modulez, r -Ztitle_underbar)r
ValueErrorrK keysr: KeyErrorrv r r( r) r r* rd r valuesr$ AttributeErrorr r r+ rC r r
r, SCHEMA_DOC_TMPL) r% rw r/ Z
required_keysZ
optional_keysZ
error_messageZ meta_copyr templater r r" get_meta_doc sd
r5 c C s t jjt jjt} t| S )N)osr7 dirnameabspath__file__r )Z configs_dirr r r" get_modulesD s r: )requested_modulesr6 c C s d}t t j dg }t| jt|}|rLtdjt |dj|dd xR|D ]J}d| ksf|| krRtj |dgdg\}}|rRtj
|d }||j pd7 }qRW |S )
zaLoad module docstrings
Docstrings are generated on module load. Reduce, reuse, recycle.
r allz+Invalid --docs value {}. Must be one of: {}z, T)r zcloudinit.configr% r ) rJ r: r1 rK
differencer r: rC r find_module
import_modulerS )r; docsZ all_modulesZ invalid_docsZmod_nameZmod_locsr modr r r" load_docI s"
rB c C s t jjt jjt jjtdS )NZschemas)r6 r7 rC r7 r8 r9 r r r r" get_schema_dird s rC )r r6 c C sZ t jjt t| d }d}ytjt|}W n& tt fk
rT t
j d| | i S X |S )ziReturn jsonschema for a specific type.
Return empty schema when no specific schema file exists.
r Nz;Skipping %s schema valiation. No JSON schema file found %s.) r6 r7 rC rC SCHEMA_FILES_BY_TYPEjsonloadsr IOErrorOSErrorr r )r Z schema_file full_schemar r r" r h s r c C s | st jddd} | jdddd | jdd td
d gd d
| jddtdt jd d | jddddd | jddddd | jd ddd d | S ) z0Return a parser for supported cmdline arguments.zcloudconfig-schemazSchema validation and documentation of instance-data configuration provided to cloud-init. This includes: user-data, vendor-data and network-config)progr. z-cz
--config-filez@Path of the cloud-config or network-config YAML file to validate)helpz-tz
--schema-typez cloud-configznetwork-configz_When providing --config-file, the schema type to validate config against. Default: cloud-config)rp choicesrK z-iz--instance-datazbPath to instance-data.json file for variable expansion of '##template: jinja' user-data. Default:
instance_data)rp rK z--system
store_trueFzVValidate the system instance-data provided as vendor-data user-data and network-config)actiondefaultrK z-dz--docs+zCPrint schema module docs. Choices: all or space-delimited cc_names.)nargsrK z
--annotatez;Annotate existing instance-data files any discovered errors)argparseArgumentParser add_argumentr* r get_runpath)parserr r r"
get_parser} sF rX c ) C sJ |j |j|jg}tdd |D dkr2tddd t|j|jgrP|jrPtd |jrh|jrhtddd t d d
}|jrtt
|j d S yt d d
}W nf t t
fk
r } z$|jtkrtjd t }n W Y d d }~X n$ tk
r t }tjd Y nX |jr|j}n$tj dkr*|jd}n
|jd}|j rL|j|j ff}ntj dkrftddd |jd}|stddd d S y2tj|jdkr|jd} tj| jr| }W n tk
r Y nX d|ff}d|jdfd|jdfd |jd ff}
x4|
D ],\} } | rtj j | r|| | ff7 }qW tj j |d d sltd |d d d d dd! d"}
t t|dk}|rtd#d$j d%d& |D d'}
g }xtt |dD ]d\}\} } d(}|rtd)| d*| d+| d, | d krt | }| }n|}| d krdn| } d }yt!| |||j|}W n t"k
r } zN| sRd-} |jst|
d.| d/| tt#||
d0 d1 |j$| W Y d d }~X n| t%k
r } z6t|
d.| tt#||
d0 d1 |j$| W Y d d }~X n*X |r|j r| n| }t|
d2| qW |rFtd$j d3d& |D d4dd! d S )5z@Handle provided schema args and perform the appropriate actions.c S s g | ] }|r|qS r r )r argr r r" r# s z&handle_schema_args..r z;Expected one of --config-file, --system or --docs argumentsT)r z[WARNING: The --schema-type parameter is inapplicable when either --system or --docs presentz;Invalid flag combination. Cannot use --annotate with --docsz cloud-config)r NZtrust)Zfetch_existing_datasourcez=Using default instance-data/user-data paths for non-root userzEdatasource not detected, using default instance-data/user-data paths.r rM Zinstance_data_sensitivezNUnable to read system userdata or vendordata as non-root user. Try using sudo.Z cloud_configz;Unable to obtain user data file. No instance data availableZ userdata_rawz user-dataz vendor-dataZvendor_cloud_configz vendor2-dataZvendor2_cloud_configznetwork-configZnetwork_configz Config file z does not existz Error: {})fmtr r z!Found cloud-config data types: %sz, c s s | ]\}}|V qd S )Nr )r cfg_typer r r r" r s z%handle_schema_args..z Fr z. z at :ZUNKNOWN_CONFIG_HEADERzInvalid rb z
Error: {}
)rZ z
Valid schema c s s | ]
}|V qd S )Nr )r r~ r r r" r H s zError: Invalid schema: {}
)&Z config_filer@ systemrz r r r r ri r rB r rG rH errnor r r r r rM r6 getuidrV Z get_ipathstatst_sizeFileNotFoundErrorr7 existsrQ rC r r rF r* ry r )r+ argsZexclusive_argsrI pathsr r Z config_filesZ
userdata_fileZraw_userdata_fileZsupplemental_config_filesr[ Zcfg_fileZnested_output_prefixZmulti_config_outputZ error_typesidxZperformed_schema_validationZ
cfg_schemar Zcfgr r r" handle_schema_args s
rg c C s t } td| j dS )zDTool to validate schema of a cloud-config file or print schema docs.zcloudconfig-schemar )rX rg
parse_args)rW r r r" mainN s ri __main__)FN)T)Nr FFTF)N)r FN)r )N)r )N)xrS rS rE Zloggingr6 r sysr( collectionsr collections.abcr copyr r^ r functoolsr itertoolsr r/ r r r
r r r
r r r Z cloudinitr r Zcloudinit.cmd.develr Zcloudinit.handlersr r Zcloudinit.sourcesr Zcloudinit.utilr r r rX r Z_ValidationErrorrY ExceptionZ getLoggerr' r Z VERSIONED_USERDATA_SCHEMA_FILEZUSERDATA_SCHEMA_FILEZ NETWORK_CONFIG_V1_SCHEMA_FILErD r r3 ZSCHEMA_PROPERTY_HEADERr" r# r& r' r ZDEPRECATED_PREFIXrI r/ r Ztyping_extensionsr$ r% r& rv r4 r5 rT r; r* rE r. rF r_ ro r} rQ r r r r r r r r r r] r r r r r r r r r r r r r$ r+ r, r5 r: rJ rB rC r rX rg ri exitr r r r" s (
)!' C%L% ]z < z
,
*F J
A