o x[h. @sUddlZddlZddlZddlZddlZddlZddlZddlmm Z ddl m Z ddl mZddlmZmZmZddlmZmZmZmZddlZddlmZmZmZmZmZmZddlm Z m!Z!ddl"m#Z#dd l$m%Z%m&Z&m'Z'dd l(m)Z)m*Z*dd l+m,Z,dd l-m.Z.m/Z/m0Z0m1Z1dd l2m3Z3ddl4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAddlBmCZCzddlDZDejEeDjDdejFdddZGeeHd<Wn$eIeJfyz ddlKZLeLjMjNjMZGWn eIyddZGYnwYnweOePZQdZRddiZSdZTdZUdZVdZWGddde ZXdZYeeZeHd <gd!Z[d"d#Z\d$d%Z]d&eZd'eZd(eeZfd)d*Z^d+eZd(eZfd,d-Z_e9d(eeZfd.d/Z`e9d+eZd(eeZfd0d1Zad~d(eZfd2d3Zbd4d5Zcd6d7Zdd8d9Zed(eeZfd:d;Zfegrdd?ehZTneQid@dZYeVdAeTidBdBdCZjdAdDdEgdBdFieUdGdHgdIZkdJeRgZldKZmdLZnGdMdNdNejoZpdOdPZqdQdRZrdSdTZsdUdVZtdWdXZuddYdZZve9d[d\Zwe9d]d^Zxe9eTd_d_fd`eZdaeZdbeydceyfdddeZze9d~dfdgZ{e9dhdiZ|djdkZ}e9dldmZ~e9eYfdndoZe9dpdqZe9drdsZe9dtedueyd(efdvdwZe9d(efdxdyZe9d~dzd{ZepZepejffgZd|d}ZdS)N)Enum)Path) monotonicsleeptime)AnyDictListOptional)net performancesourcesssh_utilsubputil) EventScope EventType) device_driver)NoDHCPLeaseErrorNoDHCPLeaseInterfaceErrorNoDHCPLeaseMissingDhclientError)EphemeralDHCPv4EphemeralIPv4Network)events)errorsidentityimdskvp)netlink) DEFAULT_WIRESERVER_ENDPOINTNonAzureDataSource OvfEnvXmlazure_ds_reporterazure_ds_telemetry_reporterbuild_minimal_ovf dhcp_log_cbget_boot_telemetryget_metadata_from_fabricget_system_inforeport_diagnostic_eventreport_dmesg_to_kvpreport_failure_to_fabric)UrlErrorz$6$)strlen)salt blowfish_hashcCstd)zRaise when called so that importing this module doesn't throw ImportError when ds_detect() returns false. In this case, crypt and passlib are not needed. z/crypt and passlib not found, missing dependency) ImportError)_r3C/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.pyr0@sAzure instance-idziid-AZURE-NODEz/dev/disk/cloud/azure_resourceext4z/var/lib/waagentz/dev/sr0c@s eZdZdZdZdZdZdZdS)PPSTypeNonePreprovisionedOSDiskRunningSavableUnknownN)__name__ __module__ __qualname__NONEOS_DISKRUNNINGSAVABLEUNKNOWNr3r3r3r4r8Ws r8z/sys/firmware/acpi/tables/OEM0PLATFORM_ENTROPY_SOURCE)z"/etc/netplan/90-hotplug-azure.yamlz /usr/local/sbin/ephemeral_eth.shz+/etc/udev/rules.d/10-net-device-added.rulesz#/run/network/interfaces.ephemeral.dcCs|D]9}td|r=|}t|dkr=|dd}t|dkr=|ddkr=|d|r=|dd}|dSqd S) z dev.storvsc.1.%pnpinfo: classid=32412632-86cb-44a2-9b5c-50d1417354f5 deviceid=00000000-0001-8899-0000-000000000000 pnpinfo=rdeviceid.N) splitlinesresearchsplitlen startswith) sysctl_outrKlinefieldscolumnscompsr3r3r4"find_storvscid_from_sysctl_pnpinfoks       rYcCs2|D]}t||r|}|dSqdS)z scbus0 on ata0 bus 0 scbus1 on ata1 bus 0 scbus2 on blkvsc0 bus 0 scbus3 on blkvsc1 bus 0 scbus4 on storvsc2 bus 0 scbus5 on storvsc3 bus 0 scbus-1 on xpt0 bus 0 rN)rNrOrPrQ)camcontrol_outdisk_drvrUitemsr3r3r4find_busdev_from_disks  r]rZbusdevreturncCsN|D] }t||r$|d}t|dkr$|dd}|dSqdS)z at scbus1 target 0 lun 0 (cd0,pass0) at scbus2 target 0 lun 0 (da0,pass1) at scbus3 target 1 lun 0 (da1,pass2) (rIrL,rN)rNrOrPrQrR)rZr^rUr\dev_passr3r3r4find_dev_from_busdevs     rcmacc CsZt|dkr)d|dd|dd|dd|dd|dd|ddg}|S) z1Normalize mac address with colons and lower-case. :rrI )rRjoinlowerrdr3r3r4normalize_mac_addresss >rncCsddtDS)z-Get Hyper-V NICs as normalized MAC addresses.cSs$g|]}|ddkrt|dqS)rI hv_netvscrLrn).0nr3r3r4 s   z1get_hv_netvsc_macs_normalized..)r get_interfacesr3r3r3r4get_hv_netvsc_macs_normalizedsrucsffddtD}d|vrdSt|dkr&td|ftjd|dStd|ftjdd S) z0Determine the device driver to match on, if any.cs$g|]}t|dkr|dqS)rLrIrprqirmr3r4rss z3determine_device_driver_for_mac..rorLz4Assuming driver for interface with mac=%s drivers=%r logger_funcrz=Unable to specify driver for interface with mac=%s drivers=%rN)r rtrRr)LOGdebugwarning)rddriversr3rmr4determine_device_driver_for_macs(  r~c Cs:zt|jWStjytdd||YSw)NzFailed to execute: %s )rstdoutProcessExecutionErrorrzr{rk)cmdfail_retr3r3r4execute_or_debugs rcCstddgddS)Nsysctlz dev.storvsc)rrr3r3r3r4get_dev_storvsc_sysctlrcCs tgdS)N) camcontroldevlistz-brr3r3r3r4get_camcontrol_dev_bus rcCs tddgS)Nrrrr3r3r3r4get_camcontrol_devrrc Csd}|dkr d}|d}dt|}d||}t}t||}|s)tddSt}t}d|}t||} | rMt || } | durKtd dS| Sd |} t|| } | rkt || } | duritd dS| SdS) N00000000rL00000001rI000z{0}-{1}z#Fail to find storvsc id from sysctlz blkvsc{0}zFail to find /dev/daXz storvsc{0}) strformatrrYrzr{rrr]rc) port_idg0g1g0g1rT storvscidcamcontrol_b_outrZblkvscscbusxdevnamestorvscr3r3r4get_resource_disk_on_freebsds<            rz freebsd-ufsrLzresource disk is not None/dev/zresource disk is None ephemeral0T)data_dir disk_aliasesapply_network_config&apply_network_config_for_secondary_ipsgptd) table_typelayout overwritez ephemeral0.1) filesystemdevice) disk_setupfs_setup datasourcenever_destroy_ntfsREDACTEDc s<eZdZdZejejejhiZ dZ e j Z dZddZdeddffd d Zd d Zd dZededefddZeddddddeededededef ddZed{ddZdefddZd{dd Zed!d"Zed#edefd$d%Z d|fd'd( Z!ed)d*Z"ed+d,Z#d-d.Z$d/d0Z%ede&efd1d2Z'd3e(de&efd4d5Z)de&efd6d7Z*d8d9Z+d:d;Z,d}dd?Z.e Ad~dBedCede/fdDdEZ0edFdGZ1edddHdIedJeddfdKdLZ2edMdNZ3ede4j4fdOdPZ5edQdRZ6ed{dSdTZ7edUdVZ8edWdXZ9ede:fdYdZZ;e dd[eedd_d`ee&edee&efdadbZ?d3e(deefdcddZ@dee(d3e(deAfdfdgZBedhdiZCedje(d3e(dee&efdkdlZDdmdnZEedodpZFeGdqdrZHedsdtZIeGdudvZJeGdwdxZKed3e(defdydzZLZMS)DataSourceAzurer5FrLcCstj||||tj|jd|_i|_d|_t t |t it g|_d|_d|_d|_d|_d|_t|_tj|jdd|_dS)NazureFdatareported_ready)r DataSource__init__ospathrkseed_dircfgseedr mergemanydictget_cfg_by_path DS_CFG_PATHBUILTIN_DS_CONFIGds_cfg_iso_dev_network_config_ephemeral_dhcp_ctx_route_configured_for_imds _route_configured_for_wireserverr_wireserver_endpoint cloud_dir_reported_ready_marker_file)selfsys_cfgdistropathsr3r3r4rDs  zDataSourceAzure.__init__ci_pkl_versionr_NcsDt|d|_d|_d|_d|_t|_tj |j j dd|_ dS)NFrr)super _unpicklerrrrrrrrrkrrr)rr __class__r3r4rVs   zDataSourceAzure._unpicklecCstj|}d||jfS)Nz %s [seed=%s])r r__str__r)rrootr3r3r4rbs zDataSourceAzure.__str__cCsH|jdurd}n|jdrd}n |jdkrd}nd}d||jfS)z/Return the subplatform metadata source details.Nunknownz/devz config-diskrzseed-dirz%s (%s))rrSrl)rsubplatform_typer3r3r4_get_subplatformfs  z DataSourceAzure._get_subplatformephipv4cCs<|jsdSdd|jD}d|v|_|j|v|_|jp|jS)NFcSsg|] }|dddqS)r/)rQ)rqrr3r3r4rsxsz5DataSourceAzure._check_if_primary..z169.254.169.254) static_routesrrr)rrrouted_networksr3r3r4_check_if_primaryrs z!DataSourceAzure._check_if_primaryT)ifacereport_failure_if_not_primary retry_sleeptimeout_minutesrrrrc Cs|jdur tdtd|tftjdt|j|t d|_d}t }||d}t j ddt d |durz|j}Wnty]td tjd|jtjt |d d d Ynotyntdtjdd|_tytd|tjd|jtjt ||dd d Yn>ty}ztd|tjdWYd}~n(d}~wtjy}ztd|j|j|j|jftjdWYd}~nd}~ww|durt ||krt|nn|dus8|durd|_t|d}||j_ d|vr|d|_!t"|} |jj#} | dur td|$| } td|| | | j%| j&||j'|j(ftjd|r>| s>|jtj)|| | j%| j&|dd d | WdS1sKwYdS)aSetup ephemeral networking. Keep retrying DHCP up to specified number of minutes. This does not kill dhclient, so the timeout in practice may be up to timeout_minutes + the system-configured timeout for dhclient. :param timeout_minutes: Number of minutes to keep retrying for. :raises NoDHCPLeaseError: If unable to obtain DHCP lease. :returns: True if NIC is determined to be primary. Nz/Bringing up networking when already configured.z2Bringing up ephemeral networking with iface=%s: %rrx)r dhcp_log_func<zobtain-dhcp-leasezobtain dhcp leasename descriptionparentzInterface not found for DHCP)durationT host_onlyzdhclient executable not foundz&Failed to obtain DHCP lease (iface=%s))r interfacezFile not found during DHCP %rz7Command failed: cmd=%r stderr=%r stdout=%r exit_code=%srz unknown-245zdhcp context missing ephipv4z{Obtained DHCP lease on interface %r (primary=%r driver=%r router=%r routes=%r lease=%r imds_routed=%r wireserver_routed=%r))rdriverrouterrlease)*r RuntimeErrorr)r rtrzr{rrr%rrReportEventStackr" obtain_leaserr|_report_failurer$ReportableErrorDhcpInterfaceNotFoundrerrorrReportableErrorDhcpLeaseFileNotFoundErrorrrrstderrr exit_coderrrr_ephipv4rrrrr(ReportableErrorDhcpOnNonPrimaryInterface) rrrrrr start_timedeadlinerrrprimaryr3r3r4_setup_ephemeral_networkings          :      &z+DataSourceAzure._setup_ephemeral_networkingcCs.d|_d|_|jdur dS|jd|_dS)zTeardown ephemeral networking.FN)rrr clean_networkrr3r3r4_teardown_ephemeral_networkings    z.DataSourceAzure._teardown_ephemeral_networkingcCs|jdup |jjdu S)z"Check if networking is configured.N)rrrr3r3r4_is_ephemeral_networking_up*s  z+DataSourceAzure._is_ephemeral_networking_upc Cszgd}t|\}}td|||ftjdWdStjye}z>t|jtr8tdt }| |n"td|j |j |j|jftjdt |}| |WYd}~dSWYd}~dSd}~ww)abCheck if azure-proxy-agent is ready for communication with WS/IMDS. If ProvisionGuestProxyAgent is true, query azure-proxy-agent status, waiting up to 120 seconds for the proxy to negotiate with Wireserver and configure an eBPF proxy. Once azure-proxy-agent is ready, it will exit with code 0 and cloud-init can then expect to be able to communicate with these services. Fail deployment if azure-proxy-agent is not found or otherwise returns an error. For more information, check out: https://github.com/azure/guestproxyagent )zazure-proxy-agentz--statusz--wait120z1Executing %s resulted in stderr=%r with stdout=%rrxzGFailed to activate Azure Guest Proxy Agent: azure-proxy-agent not foundzgFailed to activate Azure Guest Proxy Agent: status check failed cmd=%r stderr=%r stdout=%r exit_code=%sN)rr)rzr{r isinstancereasonrrr!ReportableErrorProxyAgentNotFoundrrrrr&ReportableErrorProxyAgentStatusFailure)rrouterrr report_errorreportable_errorr3r3r4_check_azure_proxy_agent_status1s@     z/DataSourceAzure._check_azure_proxy_agent_statusc Csi}|jd}d}ddi}ddddiii}d}i}t|j|D]`}z:|drFtr8tj|td d \}}}}n t|t\}}}}||_nt|\}}}}|}t d |t j d Wn-t ymt d |t j d Yqtj yt d|t j d Yqwd} t | t jd t|jp|du} | rdnd} z|j| dWn tyYnwi} |r|dr||jdd} | s|durd} t | t| ||| } | tjkrVtrd} t | t jd t| |sd} t | t jd t| | tjkr |n| tjkr|n| tj kr"|!n|"|#\}}}}|dr8||jdd} |$| }|dtjj%fvrV|&t'j(d|d|j)| d|p`d|_*|+||t,|d| ig|dt-| }t.| }t/| }|rt d|||ddd<|rt d |||d!d<|durt d"|| |d#d$<|j*dkr|d%szt0|||d&}d'|i|d%<Wnt1y}zt d(|t j d WYd}~nd}~ww|st2| }|rt d)zt34d5|6|d*<Wnt1yt d+t jd Ynw||kr%t d,|t j d t7}|r1||d!d-<|8|d!d.<|j9d/urr|rr|j:|| d0}z|j;|d1}Wn t1y^Y|Swt d2||rn||d!d3<|<|S)4aWalk all instance metadata sources returning a dict on success. @return: A dictionary of any metadata content for this instance. @raise: InvalidMetaDataException when the expected metadata service is unavailable, broken or disabled. rNlocal-hostnamer system_info default_userrrudf)mtypez!Found provisioning metadata in %srxz$Did not find Azure data source in %sz%s was not mountablezUnable to find provisioning media, falling back to IMDS metadata. Be aware that IMDS metadata does not support admin passwords or custom-data (user-data only).rrProvisionGuestProxyAgentT)report_failurezNo OVF or IMDS availablez%Free BSD is not supported for PPS VMszDHCP failed while in source PPSzextended.compute.ppsType)keyvalue)imds_mdIMDSr)rfilesmetadata userdata_rawz Username retrieved from IMDS: %sz Hostname retrieved from IMDS: %srz(Disable password retrieved from IMDS: %sr ssh_pwauthr)usernamehostname disableSshPwd ovf-env.xmlz)Failed to construct OVF from IMDS data %szRetrieved userdata from IMDSrzBad userdata in IMDSzusing files cached in %s random_seedr6F)rr pubkey_infoznegotiating returned %s public-keys)=rlist_possible_azure_dsrrSr is_FreeBSDmount_cbload_azure_ds_dirrr)rzr{r MountFailedErrorr|boolrrrgetr get_metadata_from_imdsr InvalidMetaDataException_determine_pps_typer8rArrC_wait_for_pps_running_reuserD_wait_for_pps_savable_reuserB_wait_for_pps_os_disk_shutdown_wait_for_pps_unknown_reuse _reprovision_ppstype_from_imdsrrr"ReportableErrorImdsInvalidMetadatavalidate_imds_network_metadatarupdater_username_from_imds_hostname_from_imds_disable_password_from_imdsr$ Exception_userdata_from_imdsbase64 b64decoderkrQ_get_random_seed_iid _negotiated!_determine_wireserver_pubkey_info _report_ready_cleanup_markers)r crawled_dataddir ovf_sourcemdrrrsrcmsgrequires_imds_metadatarrpps_type imds_ppstype imds_username imds_hostnameimds_disable_passwordcontentse imds_userdatarr$ssh_keysr3r3r4crawl_metadatags>                                        zDataSourceAzure.crawl_metadatarc Cst}|d}|js d}nd}d}d}ztj||dWStyI}z"t|}t|}tj||d}|js?t|j t j r?d}WYd}~n d}~wt yd}zt|}tj |d}WYd}~nd}~ww|j|| dtd|tjd iS) Ni, )max_connection_errorsretry_deadline) exceptionrF)rZrz!Ignoring IMDS metadata due to: %srx)rrr fetch_metadata_with_api_fallbackr,rrReportableErrorImdsUrlErrorrcauserequestsConnectionError ValueError+ReportableErrorImdsMetadataParsingExceptionrr)rzr|) rrrrYrX error_string error_reportrrr3r3r4r-?sH z&DataSourceAzure.get_metadata_from_imdsr3cstt||tj|_dS)z.Reset any cached class attributes to defaults.N)rrclear_cached_attrsr UNSET_metadata_imds)r attr_defaultsrr3r4rdms z"DataSourceAzure.clear_cached_attrscCs4tj}|dur dS|jdurdSt|jdS)zQCheck platform environment to report if this datasource may run. NTFr!)rChassisAssetTag query_systemrrexists)r chassis_tagr3r3r4 ds_detectrs  zDataSourceAzure.ds_detectc CsztWnty}z td|WYd}~nd}~wwztWnty;}z td|WYd}~nd}~wwzLz|}Wn?tjya}z||WYd}~W| dSd}~wty}zt |}||WYd}~W| dSd}~wwW| n| w|j r|j j dkr|j drtt}tj|rtd|tjdt|d tg|_ntd |tjd|d |_|d d |_t|d tg|_|d |_t|j d|ddddS)zCrawl and process datasource metadata caching metadata as attrs. @return: True on success, False on error, invalid or disabled datasource. z Failed to get boot telemetry: %sNz$Failed to get system information: %sFubunturzXEphemeral resource disk '%s' exists. Merging default Azure cloud ephemeral disk configs.rxrzdEphemeral resource disk '%s' does not exist. Not merging default Azure cloud ephemeral disk configs.rrrrri)dirmodeT)r&r<rzr|r(rVrReportableErrorrr!ReportableErrorUnhandledExceptionrrrr,*maybe_remove_ubuntu_network_config_scriptsRESOURCE_DISK_PATHrrrjr)r{rr#BUILTIN_CLOUD_EPHEMERAL_DISK_CONFIGrrfDEFAULT_METADATArr write_files)rrSrFrr devpathr3r3r4 _get_datasz                  zDataSourceAzure._get_datacCs&|jrd|jvr |St|jdS)Nr6)rrArrr3r3r4get_instance_idszDataSourceAzure.get_instance_idcCs|jd|S)Nr)rr,)rrr3r3r4device_name_to_devicerz%DataSourceAzure.device_name_to_devicec Cs0z ||jdWSttfyY|Sw)z+ Retrieve public SSH keys. r)_get_public_keys_from_imdsrKeyErrorr`_get_public_keys_from_ovfrr3r3r4get_public_ssh_keyss z#DataSourceAzure.get_public_ssh_keysrcCsz dd|ddD}Wntyd}t|tjdwtdd|Dr5d }t|tjdt|d t|}t|tjd|S) zGet SSH keys from IMDS metadata. :raises KeyError: if IMDS metadata is malformed/missing. :raises ValueError: if key format is not supported. :returns: List of keys. cSg|]}|dqS)keyDatar3)rq public_keyr3r3r4rssz>DataSourceAzure._get_public_keys_from_imds..compute publicKeysz"No SSH keys found in IMDS metadatarxcss|] }t|d VqdS))rN)_key_is_openssh_formatted)rqrr3r3r4 sz=DataSourceAzure._get_public_keys_from_imds..zKey(s) not in OpenSSH formatzRetrieved {} keys from IMDS)r{r)rzr{anyr`rrR)rrrUlog_msgr3r3r4rzs   z*DataSourceAzure._get_public_keys_from_imdscCsXg}z|jd}dt|}t|tjdW|Sty+d}t|tjdY|Sw)zYGet SSH keys that were fetched from wireserver. :returns: List of keys. r%zRetrieved {} keys from OVFrxzNo keys available from OVF)rrrRr)rzr{r{)rrUrr3r3r4r|s  z)DataSourceAzure._get_public_keys_from_ovfcCs|jSNrrr3r3r4get_config_obj szDataSourceAzure.get_config_objcCst|Sr)r instance_id_matches_system_uuidrx)rrr3r3r4check_instance_id sz!DataSourceAzure.check_instance_idcCsZtj|jdd}t}tj|r+t | }t |}| ||fvr+|S|S)Nrr6) rrrkr get_cpathrquery_system_uuidrjrload_text_filestripbyte_swap_system_uuidrl)rprevious prev_iid_path system_uuid swapped_idr3r3r4rAs   zDataSourceAzure._iidc Csz;d}tjddtd t|}Wdn1swY|dur0d}t|tjdWdStd|tjdWdSt yQ}z tt |tj dd}~ww)zUse the netlink socket provided to wait for nic detach event. NOTE: The function doesn't close the socket. The caller owns closing the socket and disposing it safely. Nzwait-for-nic-detachzwait for nic detachrzHPreprovisioned nic not detached as expected. Proceeding without failing.rxz%The preprovisioned nic %s is detached) rrr"rwait_for_nic_detach_eventr)rzr|r{AssertionErrorrr)rnl_sockifnamerKrr3r3r4_wait_for_nic_detach!s,  z$DataSourceAzure._wait_for_nic_detachr皙?rretriescCsft|D]}|jj|rtd|tjddS|d|kr#t|qtd|||ftjddS)NzThe link %s is up.rxrLz;The link %s is not up after %f seconds, continuing anyways.)ranger networkingtry_set_link_upr)rzinfor)rrrrrwr3r3r4wait_for_link_upAs     z DataSourceAzure.wait_for_link_upcCs@|j}td|t|djttdt dtj ddS)Nz*Creating a marker file to report ready: %sz{pid}: {time} )pidrzRSuccessfully created reported ready marker file while in the preprovisioning pool.rx) rrzrr write_filerrgetpidrr)r{)rrr3r3r4_create_report_ready_markerUs  z+DataSourceAzure._create_report_ready_marker) create_markerexpect_url_errorrrc Csz|Wn3ty9}z'|r t|tr tdtjdd|_nd}t|tjdt ||WYd}~nd}~wwd|_ |rE| dSdS)zReport ready for PPS, creating the marker file upon completion. :raises sources.InvalidMetaDataException: On error reporting ready. z,Ignoring http call failure, it was expected.rxNz9Failed reporting ready while in the preprovisioning pool.F) rDr<rr,r)rzr{rrr r.rBr)rrrrrKr3r3r4_report_ready_for_ppsbs(    z%DataSourceAzure._report_ready_for_ppsc Cstdz[g}d} d}tjddt|tdt||}Wdn1s*wY||t d|tjd | ||sRtd ||j |d dd }|r\td WdS| q t y|}zt t|tjd WYd}~dSd}~ww)z6Wait until the primary nic for the vm is hot-attached.z*Waiting for primary nic to be hot-attachedFTNzwait-for-nic-attachz4wait for nic attach after %d nics have been attachedrzDetected nic %s attached.rxz!Checking if %s is the primary nicr)rrrzFound primary nic for this VM.)rzrrrrRr"rwait_for_nic_attach_eventappendr)rrrrrr)rr nics_foundprimary_nic_foundrrr3r3r4"_wait_for_hot_attached_primary_nicsL      * z2DataSourceAzure._wait_for_hot_attached_primary_nicc Cs<ztWStjy}z td|tjdd}~ww)Nz!Failed to create netlink socket: rx)rcreate_bound_netlink_socketNetlinkCreateSocketErrorr)rzr)rrr3r3r4_create_bound_netlink_sockets z,DataSourceAzure._create_bound_netlink_socketcCs*|jddtdtjdtdt)z4Report ready and wait for host to initiate shutdown.F)rz"Waiting for host to shutdown VM...rxi3)rr)rzrrrReportableErrorOsDiskPpsFailurerr3r3r4r2s z.DataSourceAzure._wait_for_pps_os_disk_shutdownc Cs|}z_|jdus|jjdurtd|jj}|td|tjddt d,zt ||Wnt yN}zt d|tjdWYd}~nd}~wwWdn1sYwYW|n|w|dS) z9Report ready and wait for nic link to switch upon re-use.Nzmissing ephemeral contextz#Wait for vnetswitch to happen on %sz!wait-for-media-disconnect-connectzwait for vnet switchrz'Error while waiting for vnet switch: %srx)rrrrrrzr{rrr"r!wait_for_media_disconnect_connectrr)rcloser)rrrrSr3r3r4r0s<   z+DataSourceAzure._wait_for_pps_running_reusec Cs|}z=|jddz|Wntjy0}ztd|tjdd|_WYd}~nd}~ww| || |W| dS| w)z@Report ready and wait for nic(s) to be hot-attached upon re-use.T)rzLIgnoring failure while tearing down networking, NIC was likely unplugged: %rrxN) rrrrrr)rzrrrrr)rrrSr3r3r4r1s$    z+DataSourceAzure._wait_for_pps_savable_reusecCs&tj|jr dS||dS)z0Report ready if needed for unknown/recovery PPS.N)rrisfilerrrrr3r3r4r3s z+DataSourceAzure._wait_for_pps_unknown_reusec Csd}d}|sT|s |d7}z|jddWn tyYqwtjddtd"zt}WntyB| YWdqwWdn1sMwY|rt d |t j d |S) z0Poll IMDs for reprovisiondata XML document data.rNrLrrzget-reprovision-data-from-imdszget reprovision data from imdsrz#attempted dhcp %d times after reuserx) rrrrrr"rfetch_reprovision_datar,rr)rzr{)r dhcp_attemptsreprovision_datar3r3r4 _poll_imds s<   zDataSourceAzure._poll_imdsrrc Cs&td|tjdtt|}|r|S|rLztdtjdt |j |dd|_ WdSt yK}ztd|tjdWYd}~nd}~wwz*tdtjd| z|jd d Wn tyiYnwt |j |dd|_ WdSt y}ztd |tjdWYd}~d Sd}~ww) aReport failure to Azure host and fabric. For errors that may be recoverable (e.g. DHCP), host_only provides a mechanism to report the failure that can be updated later with success. DHCP will not be attempted if host_only=True and networking is down. @param error: Error to report. @param host_only: Only report to host (error may be recoverable). @return: The success status of sending the failure signal. z#Azure datasource failure occurred: rxz>Using cached ephemeral dhcp context to report failure to Azure)endpointrTz@Failed to report failure using cached ephemeral dhcp context: %sNz3Using new ephemeral dhcp to report failure to Azurerrz5Failed to report failure using new ephemeral dhcp: %sF)r)as_encoded_reportrzrr*rreport_failure_to_hostrr{r+rrBr<rrr)rrrreportedrSr3r3r4r>sh   zDataSourceAzure._report_failurer#r$c Csfttz t|j|j|j|d}Wnty*}z td|t j dd}~wwd|_d|_ |S)zTells the fabric provisioning has completed. :param pubkey_info: Fingerprints of keys to request from Wireserver. :raises Exception: if failed to report. :returns: List of SSH keys, if requested. )rriso_devr$zQError communicating with Azure fabric; You may experience connectivity issues: %srxNT) r*rreport_success_to_hostr'rrrr<r)rzr|rB)rr$rrSr3r3r4rDs,   zDataSourceAzure._report_readyc CsLz |dddWSty%}ztd|tjdWYd}~dSd}~ww)NextendedrppsTypez2Could not retrieve pps configuration from IMDS: %srx)r<r)rzr{)rrrSr3r3r4r5sz"DataSourceAzure._ppstype_from_imdsovf_cfgcCstj|jr tj}nO|ddtjjks| |tjjkr"tj}n8|ddtj jks5| |tj jkr9tj }n!|ddusS|ddtj jksS| |tj jkrWtj }ntj }t d|jtjd|S)z@Determine PPS type using OVF, IMDS data, and reprovision marker.PreprovisionedVMTypeNPreprovisionedVmTz PPS type: %srx)rrrrr8rEr,rDrr5rBrCrAr)rzr)rrrrMr3r3r4r/s&  z#DataSourceAzure._determine_pps_typecCsZ|}tjddtdt|\}}}|||d|ifWdS1s&wYdS)zrInitiate the reprovisioning workflow. Ephemeral networking is up upon successful reprovisioning. zreprovisioning-read-azure-ovfz$read azure ovf during reprovisioningrr!N)rrrr"read_azure_ovf)rrRrIudrr3r3r4r4s$zDataSourceAzure._reprovisionrc Csbd}z||W|Sttfy0|dd}d|dur#t|nd}t|tjdY|Sw)zDetermine the fingerprints we need to retrieve from Wireserver. :return: List of keys to request from Wireserver, if any, else None. N_pubkeysz"Retrieved {} fingerprints from OVFrrx) rzr{r`r,rrRr)rzr{)rrrr$rr3r3r4rCs  z1DataSourceAzure._determine_wireserver_pubkey_infocCst|jdS)zCleanup any marker files.N)rdel_filerrr3r3r4rEsz DataSourceAzure._cleanup_markerscCs8|j}zt|||jtddWtdStw)NF)is_new_instance preserve_ntfs)r get_ipath_curaddress_ephemeral_resizerr,DS_CFG_KEY_PRESERVE_NTFSr*)rrr instance_dirr3r3r4activates  zDataSourceAzure.activatecC|jdididS)NrrplatformFaultDomainrr,rr3r3r4availability_zones z!DataSourceAzure.availability_zonec Cs|jr8|jtjkr8|jdr8zt|jd|jddWSty7}ztdt |WYd}~nd}~wwzt WStyW}ztdt |WYd}~iSd}~ww)z:Generate network configuration according to configuration.rnetworkr)rz?Failed generating network config from IMDS network metadata: %sNz-Failed generating fallback network config: %s) rfr rerr,6generate_network_config_from_instance_network_metadatar<rzrr-_generate_network_config_from_fallback_config)rrSr3r3r4_generate_network_configs4  z(DataSourceAzure._generate_network_configcCs(|jr |jtjkr |jS||_|jS)z,Provide network configuration v2 dictionary.)rr rerrr3r3r4network_config$s zDataSourceAzure.network_configcCr)Nrrlocationrrr3r3r4region.szDataSourceAzure.regioncst}z|d}dd|dDWnty(td|dtjdYdSwfdd|D}|s6d Std ||ftjd|jrH|jjsJdSt |jj}|rXt |t sZdSt |}||vrmtd ||ftjddS) z=Validate IMDS network config and report telemetry for errors.rcSsg|]}t|dqS) macAddressrprvr3r3r4rs9s zBDataSourceAzure.validate_imds_network_metadata..rz6IMDS network metadata has incomplete configuration: %rrxFcsg|]}|vr|qSr3r3)rqm imds_macsr3r4rsEsTz>IMDS network metadata is missing configuration for NICs %r: %rz3IMDS network metadata is missing primary NIC %r: %r) rur{r)r,rzr|rrr get_interface_macrrrn)rr local_macsr missing_macs primary_macr3rr4r72sL z.DataSourceAzure.validate_imds_network_metadata)r_N)r3r)rr)F)Nr>r?r@dsnamerNETWORKrBOOT_NEW_INSTANCEBOOTdefault_update_eventsrBr rerf_ci_pkl_versionrintrrrr#rr+rr rrrrr rVrr-rdrlrwrxryr r}dictrzr|rrrArfloatrrrrsocketrr2r0r1r3bytesrrrorrDr5r8r/r4rCrErpropertyrrrrr7 __classcell__r3r3rr4r8s      6 X-  H       & 8  $    A   "         rcC(z |dddWStyYdSw)Nr osProfile adminUsernamer{ imds_datar3r3r4r9c  r9cCs$z|ddWStyYdSw)NruserDatarrr3r3r4r=js  r=cCr)Nrr computerNamerrr3r3r4r:qrr:cCs,z |ddddkWStyYdSw)NrrdisablePasswordAuthenticationtruerrr3r3r4r;xs r;cCsFd|vrdSt}z||}Wn tyYdSw|jduS)z? Validate whether or not the key is OpenSSH-formatted. z FN)rrAuthKeyLineParserparse TypeErrorkeytype)rparseraklr3r3r4rs   rcCs`dD]+}g}td|D]}||t|}tj|r&||tj|fq |r-|SqgS)N)z-partprrL)rrrrrjrrealpath)rvmaxnumsufffoundpnumppathr3r3r4_partitions_on_devices rcCs*tjddd}td|tj||vS)Nz TYPE=ntfsT)no_cachezntfs_devices found = %s)rfind_devs_withrzr{rrr)rv ntfs_devicesr3r3r4_has_ntfs_filesystems r c Cs|rddttf}d|fStj|sdd|fStd|tj|t |}|s2dd|fSt |}|dkrLd|d d d |Df}d|fS|dkrW|d \}}n|d \}}t |sld|||f}d|fSt dd}d|||f}t jddtdy} ztj||dddidd} Wn@tjy} z3d| _dt| vrd|dfWYd} ~ WdSd|d|| ffWYd} ~ WdSd} ~ ww| d krd| | _tddttd|d | fWdSWdn1swYd|d!fS)"aDetermine if the ephemeral drive at devpath should be reformatted. A fresh ephemeral disk is formatted by Azure and will: a.) have a partition table (dos or gpt) b.) have 1 partition that is ntfs formatted, or have 2 partitions with the second partition ntfs formatted. (larger instances with >2TB ephemeral disk have gpt, and will have a microsoft reserved partition as part 1. LP: #1686514) c.) the ntfs partition will have no files other than possibly 'dataloss_warning_readme.txt' User can indicate that NTFS should never be destroyed by setting DS_CFG_KEY_PRESERVE_NTFS in dscfg. If data is found on NTFS, user is warned to set DS_CFG_KEY_PRESERVE_NTFS to make sure cloud-init does not accidentally wipe their data. If cloud-init cannot mount the disk to check for data, destruction will be allowed, unless the dscfg key is set.z:config says to never destroy NTFS (%s.%s), skipping checksrMFzdevice %s does not existzResolving realpath of %s -> %szdevice %s was not partitionedrIz&device %s had 3 or more partitions: %srcSr~)rLr3)rqrr3r3r4rssz*can_dev_be_reformatted..rLrz5partition %s (%s) on device %s was not ntfs formattedcs(tddgtfddt|DS)Nzdataloss_warning_readme.txtzsystem volume informationcsg|] }|vr|qSr3)rl)rqfignoredr3r4rssz?can_dev_be_reformatted..count_files..)setrRrlistdir)mpr3r r4 count_filessz+can_dev_be_reformatted..count_filesz1partition %s (%s) on device %s was ntfs formattedzmount-ntfs-and-countrntfsLANGC)rupdate_env_for_mount log_errorzcannot mount ntfszunknown filesystem type 'ntfs'Tz^ but this system cannot mount NTFS, assuming there are no important files. Formatting allowed.Nz but mount of %s failed: %szmounted and counted %d fileszxit looks like you're using NTFS on the ephemeral disk, to ensure that filesystem does not get wiped, set %s.%s in configz but had %d files on it.z3 and had no important files. Safe for reformatting.)rkrrrrrjrzr{rrrRr r#rrr"rr(r*rrr|) rvrrK partitions partition_len cand_part cand_pathrbmsgevt file_countrSr3r3r4can_dev_be_reformatteds          %rFrrvrrc Cstj|std|tjddStd|tjdd}d}|r%d\}}nt||\}}td|||s7dSdD]E}tj|dd |}d ||f}tj|rxz t|td |Wq9t yw} z t d || WYd} ~ q9d} ~ wwtd |q9dS)Nz,Ephemeral resource disk '%s' does not exist.rxz$Ephemeral resource disk '%s' exists.F)TzFirst instance boot.zreformattable=%s: %s)rmountssemconfig_zMarker "%s" for module "%s"z %s removed.z%s: remove failed! (%s)z%s did not exist.) rrrjr)rzr{rrkunlinkrr|) rrvrrresultrKmodsempathrrSr3r3r4rs>     rcCsjdd}|sdS|s i}t|||D]\}}tj||}d|vr*|||}tj||ddqdS)NcSsbzt|}|D]}d|jvr|jtkrt|_q t|WSty0t d||YSw)z>Azure provides the UserPassword in plain text. So we redact it UserPasswordz#failed to redact userpassword in %s) ET fromstringitertagtextDEF_PASSWD_REDACTIONtostringr<rzcritical)cntfnamerelemr3r3r4_redact_passwordFs       z%write_files.._redact_passwordr!i)filenamecontentmode)r ensure_dirr\rrrkr)datadirrrnr2rr4r0r3r3r4ruDs  rucCst|}i}i}|jp d}|jr|j|d<|jr|j|d<|jdur*|j |d<n|jr1d|d<i}|jr;|j|d<|jrNd|d <t|jkrNt |j|d <|rVd |i|d <|j |d <t d|j t j d|j|d<t d|jt j d|j|d<t d|jt j d|||fS)zParse OVF XML contents. :return: Tuple of metadata, configuration, userdata dicts. :raises NonAzureDataSource: if XML is not in Azure's format. :raises errors.ReportableError: if XML is unparsable or invalid. rr rNrTrF lock_passwd hashed_passwdrrrzPreprovisionedVm: %srxrzPreprovisionedVMType: %srzProvisionGuestProxyAgent: %s)r! parse_text custom_datar public_keysdisable_ssh_password_authpasswordrr, encrypt_passpreprovisioned_vmr)rzrpreprovisioned_vm_typeprovision_guest_proxy_agent)rRovf_envrIrrdefuserr3r3r4rasJ            rcCst|Sr)r0)r>r3r3r4r?sr?cCs`zt|}|d WdWdS1swYWdSty/td|YdSw)z;Return boolean indicating path to cdrom device has content.iNTzcdrom (%s) is not configuredF)openreadIOErrorrzr{) cdrom_devfpr3r3r4_check_freebsd_cdroms  $ rJcCs(|durdStj|dd}t|S)zIReturn content random seed file if available, otherwise, return None.NT)quiet)rload_binary_filer> b64encodedecode)sourcerr3r3r4r@s r@ccsZ|VtVtrd}t|r|VndD] }td|EdHq|r+|VdSdS)Nz/dev/cd0)iso9660rzTYPE=%s)DEFAULT_PROVISIONING_ISO_DEVrr'rJr)r cache_dirrHfstyper3r3r4r&s r&c Cstj|d}tj|stdtd!t|d }|}Wdn1s+wYWdn1s:wYt |\}}}|||d|ifS)Nr!zNo ovf-env file foundzReading ovf-env.xmlrb) rrrkrr r TimedrErFr) source_dirovf_filerIrRrIrrr3r3r4r)s  r)network_metadatarc Csdid}t|dD]\}}d}dj|d}d|dd i}|d kr'd|d <d |dd }dD]\} || idg} | sEtd| |q/d }| dkrNd} n d} | rZd |d<||d<|s]q/| ddD]'} || dd d| } | d}|ds~g|d<|ddj|| dqcq/|r|rt|d}|d|i|dt |}|r||d d!<||d"|<q td#|||q |S)$zConvert imds network metadata dictionary to network v2 configuration. :param: network_metadata: Dict of "network" key from instance metadata. :return: Dictionary containing network version 2 standard configuration. rI)version ethernetsrFzeth{idx})idxz route-metricrLrrzuse-dnsT)dhcp4zdhcp4-overridesdhcp6)ipv4ipv6 ipAddresszNo %s addresses found for: %rr^24128r]zdhcp6-overridesNsubnetprefixprivateIpAddress addressesz {ip}/{prefix})iprdr macaddress)matchzset-namerirrZz functoolsloggingros.pathrOrxml.etree.ElementTreeetree ElementTreer'enumrpathlibrrrrtypingrrr r r^ cloudinitr r r rrrcloudinit.eventrr cloudinit.netrcloudinit.net.dhcprrrcloudinit.net.ephemeralrrcloudinit.reportingrcloudinit.sources.azurerrrrcloudinit.sources.helpersrcloudinit.sources.helpers.azurerr r!r"r#r$r%r&r'r(r)r*r+cloudinit.url_helperr,cryptpartialrand_strr0__annotations__r1AttributeError passlib.hashpasslibhash sha512_crypt getLoggerr>rzDS_NAMErtrr DEFAULT_FSAGENT_SEED_DIRrQr8rFrrxrYr]rcrnrur~rrrrrr'res_diskr{rrsrrr,rrr9r=r:r;rrr rr+rrurr?rJr@r&r)rrrrqDataSourceAzureNetDEP_FILESYSTEMr~rr3r3r3r4s&       <       +       5   p +  7      P &