{"id":45,"date":"2022-08-19T08:40:02","date_gmt":"2022-08-19T06:40:02","guid":{"rendered":"https:\/\/www.nicedata.fr\/?p=45"},"modified":"2024-10-01T15:04:28","modified_gmt":"2024-10-01T13:04:28","slug":"parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse","status":"publish","type":"post","link":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/","title":{"rendered":"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse !"},"content":{"rendered":"\n<p>Cet article \u00e9tait loin dans les cartons, mais l&rsquo;article <a rel=\"noreferrer noopener\" href=\"https:\/\/hernandezpaul.wordpress.com\/2022\/08\/10\/alternative-pipeline-parametrization-for-azure-synapse-analytics\/\" data-type=\"URL\" data-id=\"https:\/\/hernandezpaul.wordpress.com\/2022\/08\/10\/alternative-pipeline-parametrization-for-azure-synapse-analytics\/\" target=\"_blank\">Alternative pipeline parametrization for Azure Synapse Analytics<\/a> a motiv\u00e9 la priorisation de cet article. Merci <a rel=\"noreferrer noopener\" href=\"https:\/\/twitter.com\/paul_eng\" data-type=\"URL\" data-id=\"https:\/\/twitter.com\/paul_eng\" target=\"_blank\">@paul_eng<\/a> pour le partage et la motivation d&rsquo;\u00e9crire celui-ci !<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_81 ez-toc-wrap-center counter-hierarchy ez-toc-counter ez-toc-transparent ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Sommaire<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Introduction\" >Introduction<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Cas_dusage_%E2%80%93_Ingerer_un_sous-ensemble_de_donnees\" >Cas d&rsquo;usage &#8211; Ing\u00e9rer un sous-ensemble de donn\u00e9es<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Solution\" >Solution<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Les_mains_dedans\" >Les mains dedans !<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Vue_densemble\" >Vue d&rsquo;ensemble<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#La_base_de_donnees_surtout_la_table\" >La base de donn\u00e9es (surtout la table !)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Le_pipeline_et_autres_objets_necessaires\" >Le pipeline (et autres objets n\u00e9cessaires)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Premiere_execution\" >Premi\u00e8re ex\u00e9cution<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Tout_ca_%E2%80%A6_pour_ca\" >Tout \u00e7a &#8230; pour \u00e7a ?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Et_quoi_dautre\" >Et quoi d&rsquo;autre ?<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Conclusion\" >Conclusion<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#Pour_aller_plus_loin\" >Pour aller plus loin<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Introduction\"><\/span>Introduction<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Que se soit lors de l&rsquo;\u00e9criture du code d&rsquo;un site web, d&rsquo;un script ou de projet data, il est toujours important de rendre param\u00e9trable l&rsquo;ex\u00e9cution du code que l&rsquo;on \u00e9crit. La majeur parti des langages, framework, IDE ou solution packag\u00e9e utilis\u00e9e en informatique proposent tous leur propre solution pour traiter ce sujet avec plus ou moins d&rsquo;efficacit\u00e9 et d&rsquo;\u00e9l\u00e9gance.<\/p>\n\n\n\n<p>L&rsquo;utilisation de param\u00e8tres permet de g\u00e9rer certaines particularit\u00e9s dans le traitement g\u00e9n\u00e9ral, mais reviens maintenant le probl\u00e8me de comment d\u00e9finir et initialiser ces param\u00e8tres sans avoir \u00e0 modifier quoi que ce soit dans le code ou les configurations internes de chaque composant.<\/p>\n\n\n\n<p>Je n&rsquo;ai pas trouv\u00e9 dans Synapse un moyen simple et int\u00e9gr\u00e9 pour g\u00e9rer les param\u00e8tres \u00e0 l&rsquo;ext\u00e9rieur de Synapse.<\/p>\n\n\n\n<p>Dans notre \u00e9quipe nous avons commenc\u00e9 \u00e0 travailler avec des fichiers de configurations d\u00e9pos\u00e9s dans le Lake, mais tr\u00e8s vite leur manipulation s&rsquo;est av\u00e9r\u00e9e d\u00e9licate et leur multiplication n&rsquo;a pas aid\u00e9 \u00e0 rendre le tout plus facile d&rsquo;usage. <\/p>\n\n\n\n<p>Il nous fallait donc trouver quelque chose de plus \u00ab\u00a0User Friendly\u00a0\u00bb, que nous maitrisions, qui soit centralis\u00e9 et ext\u00e9rieur \u00e0 nos diff\u00e9rents codes. Dans le monde de la \u00ab\u00a0Data\u00a0\u00bb, nous parlons tous couramment le SQL (\u00e0 minima SELECT\/ INSERT \/ UPDATE) et donc la Base de donn\u00e9es de param\u00e8tre nous est apparut comme une \u00e9vidence. De plus gr\u00e2ce au cloud il est possible de monter une petite base SQL en quelques cliques pour quelques euros par mois ! <\/p>\n\n\n\n<p>Le sujet est maintenant amen\u00e9, nous devons maintenant d\u00e9finir ce que nous voulons param\u00e9trer et pourquoi !<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Cas_dusage_%E2%80%93_Ingerer_un_sous-ensemble_de_donnees\"><\/span>Cas d&rsquo;usage &#8211; Ing\u00e9rer un sous-ensemble de donn\u00e9es<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Lors de l&rsquo;alimentation d&rsquo;un Datalake, la premi\u00e8re \u00e9tape est d&rsquo;ing\u00e9rer les donn\u00e9es depuis des donn\u00e9es sources. Ces actions r\u00e9p\u00e9t\u00e9es sur l&rsquo;ensemble des sources de donn\u00e9es sont r\u00e9p\u00e9titives et il existe donc de nombreuses fa\u00e7ons d&rsquo;automatiser ce processus. Cependant, dans de nombreux cas, cette automatisation rend homog\u00e8ne la fa\u00e7on d&rsquo;ing\u00e9rer les donn\u00e9es :<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Ingestion de TOUTES les tables d&rsquo;une base de donn\u00e9es<\/li>\n\n\n\n<li>Ingestion de TOUS les fichiers d&rsquo;un r\u00e9pertoire<\/li>\n\n\n\n<li>&#8230;<\/li>\n<\/ul>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-28f84493 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p>Imaginons maintenant que dans notre base de donn\u00e9e source il y ait de nombreuses tables temporaires, de travail ou simplement non utilis\u00e9 dans les projets actuels. Si l&rsquo;on peut se dire qu&rsquo;il est plus \u00ab\u00a0simple\u00a0\u00bb d&rsquo;ing\u00e9rer l&rsquo;int\u00e9gralit\u00e9 et de trier ensuite, il existe certains cas o\u00f9 ce n&rsquo;est pas forc\u00e9ment pertinent.<\/p>\n\n\n\n<p>Imaginons que nous travaillions pour WideWorldImporters et que j&rsquo;ai pour mission de pr\u00e9senter un tableau de bord avec uniquement un suivi des articles factur\u00e9s.<\/p>\n\n\n\n<p>Je pourrais effectivement ing\u00e9rer l&rsquo;int\u00e9gralit\u00e9 de ma base WWI mais honn\u00eatement seulement 3 tables m&rsquo;int\u00e9ressent alors pourquoi ing\u00e9rer l&rsquo;int\u00e9gralit\u00e9 de ma base qui comporte de nombreuses tables non utilis\u00e9es pour notre usage ?<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:33.33%\">\n<figure class=\"wp-block-image size-full is-resized is-style-default\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"385\" height=\"686\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image.png?resize=385%2C686\" alt=\"\" class=\"wp-image-49\" style=\"width:257px;height:458px\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image.png?w=385&amp;ssl=1 385w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image.png?resize=168%2C300&amp;ssl=1 168w\" sizes=\"auto, (max-width: 385px) 100vw, 385px\" \/><figcaption class=\"wp-element-caption\">Liste des tables de la base WWI<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p>Le d\u00e9bat n&rsquo;est pas pourquoi on voudrait faire cela, mais comment mettre en place un pipeline d&rsquo;ingestion qui nous permette de param\u00e9trer les donn\u00e9es\/tables que nous souhaitons \u00e0 l&rsquo;ext\u00e9rieur de nos pipelines Synapse.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Solution\"><\/span>Solution<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Une des solutions (en tout cas celle retenue ici) est d&rsquo;utiliser une table de param\u00e9trage dans une base de donn\u00e9es d\u00e9di\u00e9e. Le pipeline aura donc pour processus d&rsquo;aller dans un premier temps chercher et initialiser les param\u00e8tres dans la table d\u00e9di\u00e9e et ensuite d&rsquo;ex\u00e9cuter ces activit\u00e9s ou sous pipeline.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"599\" height=\"429\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-1.png?resize=599%2C429&#038;ssl=1\" alt=\"\" class=\"wp-image-51\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-1.png?w=599&amp;ssl=1 599w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-1.png?resize=300%2C215&amp;ssl=1 300w\" sizes=\"auto, (max-width: 599px) 100vw, 599px\" \/><figcaption class=\"wp-element-caption\">Principe g\u00e9n\u00e9ral du pipeline<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Les_mains_dedans\"><\/span>Les mains dedans !<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Les ressources utilis\u00e9es dans cette exp\u00e9rience sont:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Un workspace Synapse (la base)<\/li>\n\n\n\n<li>Une base de donn\u00e9e accessible depuis le workspace (l&rsquo;objet de l&rsquo;exp\u00e9rience)<\/li>\n\n\n\n<li>Un Azure data lake gen 2 (ou tout autre \u00ab\u00a0r\u00e9pertoire de d\u00e9p\u00f4t\u00a0\u00bb)<\/li>\n\n\n\n<li>La base de donn\u00e9es SQL WideWorldImporters accessible depuis le workspace synapse disponible sur le github des samples Microsoft : <a href=\"https:\/\/github.com\/microsoft\/sql-server-samples\/releases\/download\/wide-world-importers-v1.0\/WideWorldImporters-Full.bak\">WideWorldImporters-Full.bak (github.com)<\/a><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Vue_densemble\"><\/span>Vue d&rsquo;ensemble<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>La solution consiste en seulement trois activit\u00e9s<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Lookup &#8211; pour la lecture du param\u00e9trage<\/li>\n\n\n\n<li>ForEach &#8211; pour l&rsquo;ex\u00e9cution \u00ab\u00a0pour chaque ligne de param\u00e9trage\u00a0\u00bb<\/li>\n\n\n\n<li>CopyData &#8211; pour l&rsquo;ex\u00e9cution de l&rsquo;ingestion<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"464\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-3.png?resize=1024%2C464&#038;ssl=1\" alt=\"Vue d'ensemble du pipeline montrant les 3 activit\u00e9s utilis\u00e9s\" class=\"wp-image-53\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-3.png?resize=1024%2C464&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-3.png?resize=300%2C136&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-3.png?resize=768%2C348&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-3.png?resize=1200%2C544&amp;ssl=1 1200w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-3.png?w=1416&amp;ssl=1 1416w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n\n\n\n<p>Ce pipeline est volontairement \u00ab\u00a0simple\u00a0\u00bb pour pr\u00e9senter uniquement l&rsquo;utilisation de la base de donn\u00e9es de param\u00e9trage.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"La_base_de_donnees_surtout_la_table\"><\/span>La base de donn\u00e9es (surtout la table !)<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>La base de donn\u00e9es est tr\u00e8s simple et comprend dans notre cas uniquement trois colonnes:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>StagingDirectoryName (le r\u00e9pertoire de destination de notre extraction dans le lake)<\/li>\n\n\n\n<li>FileName (le nom du fichier qui sera \u00e9crit dans le r\u00e9pertoire pr\u00e9c\u00e9dent)<\/li>\n\n\n\n<li>SourceQuery (la requ\u00eate d&rsquo;extraction de donn\u00e9es)<\/li>\n<\/ul>\n\n\n\n<p>Voici le DDL utilis\u00e9 pour cr\u00e9er la table et les valeurs utilis\u00e9es dans l&rsquo;exp\u00e9rience :<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code aligncenter\"><pre class=\"brush: sql; title: ; notranslate\" title=\"\">\nUSE &#x5B;SynapseTraining]\nGO\n\n-- Assure we have a proper schema\nIF NOT EXISTS ( SELECT 1 FROM sys.schemas WHERE name = N&#039;param&#039; )\n    EXEC(&#039;CREATE SCHEMA &#x5B;param]&#039;);\nGO\n\n-- Droppring and creating the IngestionParam table\nIF EXISTS ( select 1 from INFORMATION_SCHEMA.TABLES\n            where TABLE_SCHEMA = &#039;param&#039; and TABLE_NAME = &#039;IngestionParam&#039; )\n    DROP TABLE &#x5B;param].&#x5B;IngestionParam]\nGO\n\nCREATE TABLE &#x5B;param].&#x5B;IngestionParam](\n    &#x5B;StagingDirectoryName] varchar(200),\n    &#x5B;FileName] varchar(100),\n    &#x5B;SourceQuery] varchar(500)\n)\nGO\n\n-- Inserting sample values\nINSERT INTO &#x5B;param].&#x5B;IngestionParam](&#x5B;StagingDirectoryName], &#x5B;FileName], &#x5B;SourceQuery])\n    VALUES (&#039;staging\/Invoices\/&#039;,&#039;Invoices.parquet&#039;,&#039;SELECT * FROM Sales.Invoices&#039;)\n\n\n\/*\nInserting more sample value to demonstrate the ease of usability\n\nINSERT INTO &#x5B;param].&#x5B;IngestionParam](&#x5B;StagingDirectoryName], &#x5B;FileName], &#x5B;SourceQuery])\n    VALUES (&#039;staging\/InvoiceLines\/&#039;,&#039;InvoiceLines.parquet&#039;,&#039;SELECT * FROM Sales.InvoiceLines&#039;)\nINSERT INTO &#x5B;param].&#x5B;IngestionParam](&#x5B;StagingDirectoryName], &#x5B;FileName], &#x5B;SourceQuery])\n    VALUES (&#039;staging\/StockItems\/&#039;,&#039;StockItems.parquet&#039;,&#039;SELECT * FROM Warehouse.StockItems&#039;)\n*\/\nGO\n<\/pre><\/div>\n\n\n<p>Une fois ex\u00e9cut\u00e9e, notre base de param\u00e9trage est pr\u00eate :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"432\" height=\"171\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-4.png?resize=432%2C171&#038;ssl=1\" alt=\"\" class=\"wp-image-54\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-4.png?w=432&amp;ssl=1 432w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-4.png?resize=300%2C119&amp;ssl=1 300w\" sizes=\"auto, (max-width: 432px) 100vw, 432px\" \/><figcaption class=\"wp-element-caption\">R\u00e9sultat de l&rsquo;execution du script SQL<\/figcaption><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Le_pipeline_et_autres_objets_necessaires\"><\/span>Le pipeline (et autres objets n\u00e9cessaires)<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Lookup<\/h4>\n\n\n\n<p>La premi\u00e8re \u00e9tape est donc de lire notre table de param\u00e9trage et pour ce faire, on utilise une activit\u00e9 Lookup. Celle-ci utilise en dataset source la connexion \u00e0 notre base de donn\u00e9es de param\u00e9trage et une requ\u00eate tr\u00e8s simple de lecture de notre table de param\u00e9trage.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"756\" height=\"539\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-5.png?resize=756%2C539&#038;ssl=1\" alt=\"\" class=\"wp-image-55\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-5.png?w=756&amp;ssl=1 756w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-5.png?resize=300%2C214&amp;ssl=1 300w\" sizes=\"auto, (max-width: 756px) 100vw, 756px\" \/><figcaption class=\"wp-element-caption\">Configuration du lookup<\/figcaption><\/figure>\n\n\n\n<p>La pr\u00e9visualisation des donn\u00e9es extraites du Lookup doit nous remonter la ligne de donn\u00e9e provenant de notre base de param\u00e9trage.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"764\" height=\"198\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-7.png?resize=764%2C198&#038;ssl=1\" alt=\"\" class=\"wp-image-57\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-7.png?w=764&amp;ssl=1 764w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-7.png?resize=300%2C78&amp;ssl=1 300w\" sizes=\"auto, (max-width: 764px) 100vw, 764px\" \/><figcaption class=\"wp-element-caption\">Pr\u00e9visualisation des donn\u00e9es du Lookup<\/figcaption><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\">ForEach<\/h4>\n\n\n\n<p>Maintenant, ce que nous souhaitons faire c&rsquo;est \u00ab\u00a0ex\u00e9cuter la suite\u00a0\u00bb pour chaque ligne de param\u00e9trage ins\u00e9r\u00e9 dans notre base et donc renvoy\u00e9 par le Lookup. C&rsquo;est l&rsquo;activit\u00e9 ForEach qui nous le permet en prenant en entr\u00e9e chaque ligne renvoy\u00e9e par le Lookup pr\u00e9c\u00e9dent. C&rsquo;est dans l&rsquo;onglet \u00ab\u00a0Settings\u00a0\u00bb que cela se passe en pr\u00e9cisant les \u00ab\u00a0Items\u00a0\u00bb. Pour se faire, il faut ajouter du contenu dynamique et ajouter le \u00ab\u00a0value array\u00a0\u00bb de l&rsquo;activit\u00e9 pr\u00e9c\u00e9dente.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"620\" height=\"597\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-8.png?resize=620%2C597&#038;ssl=1\" alt=\"\" class=\"wp-image-58\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-8.png?w=620&amp;ssl=1 620w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-8.png?resize=300%2C289&amp;ssl=1 300w\" sizes=\"auto, (max-width: 620px) 100vw, 620px\" \/><figcaption class=\"wp-element-caption\">Param\u00e9trage des Items du ForEach<\/figcaption><\/figure>\n\n\n\n<p>Suivant le besoin il est possible d&rsquo;utiliser les autres param\u00e8tres du ForEach pour faire un traitement s\u00e9quentiel ou en batch, mais ce n&rsquo;est pas n\u00e9cessaire pour notre exp\u00e9rience.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"497\" height=\"381\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-6.png?resize=497%2C381&#038;ssl=1\" alt=\"\" class=\"wp-image-56\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-6.png?w=497&amp;ssl=1 497w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-6.png?resize=300%2C230&amp;ssl=1 300w\" sizes=\"auto, (max-width: 497px) 100vw, 497px\" \/><figcaption class=\"wp-element-caption\">Configuration du ForEach<\/figcaption><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\">Copy data<\/h4>\n\n\n\n<p>Maintenant que notre pipeline est pr\u00eat pour ex\u00e9cuter une activit\u00e9 pour chaque ligne de param\u00e9trage, nous sommes pr\u00eats pour enfin ex\u00e9cuter notre copie ! Pour se faire, on ajoute une activit\u00e9 \u00e0 l&rsquo;int\u00e9rieur du ForEach.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"244\" height=\"164\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-11.png?resize=244%2C164&#038;ssl=1\" alt=\"\" class=\"wp-image-61\"\/><figcaption class=\"wp-element-caption\">Ajout d&rsquo;une activit\u00e9 dans le ForEach<\/figcaption><\/figure>\n\n\n\n<p>Pour notre usage, nous utilisons l&rsquo;activit\u00e9 Copy data et nous avons deux choses \u00e0 configurer :<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Source &#8211; pour configurer \u00ab\u00a0Ce qui va \u00eatre extrait\u00a0\u00bb<\/li>\n\n\n\n<li>Sink &#8211; pour configurer ou et comment on stocke les donn\u00e9es pr\u00e9c\u00e9demment extraites.<\/li>\n<\/ul>\n\n\n\n<p>A ce stade de notre d\u00e9veloppement, il faut se dire que gr\u00e2ce \u00e0 notre Lookup et au ForEach, nous avons dans les param\u00e8tres de notre activit\u00e9 l&rsquo;ensemble des valeurs (colones) d&rsquo;une ligne de notre table de param\u00e9trage dans la \u00ab\u00a0variable\u00a0\u00bb @item()\u00a0\u00bb. Par exemple :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"391\" height=\"41\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-12.png?resize=391%2C41&#038;ssl=1\" alt=\"\" class=\"wp-image-62\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-12.png?w=391&amp;ssl=1 391w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-12.png?resize=300%2C31&amp;ssl=1 300w\" sizes=\"auto, (max-width: 391px) 100vw, 391px\" \/><figcaption class=\"wp-element-caption\">Exemple de valeurs disponible pour nos activit\u00e9s<\/figcaption><\/figure>\n\n\n\n<p>Au niveau de la source, une fois le dataset source connect\u00e9e \u00e0 notre base WWI configur\u00e9e, nous avons uniquement \u00e0 configurer le fait d&rsquo;utiliser une requ\u00eate que l&rsquo;on r\u00e9cup\u00e8re de notre variable @item() auquel on demande la valeur \u00ab\u00a0SourceQuery\u00a0\u00bb de notre table de param\u00e9trage. C&rsquo;est ce qui sera envoy\u00e9 sur notre serveur pour extraction de donn\u00e9es.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"789\" height=\"620\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-9.png?resize=789%2C620&#038;ssl=1\" alt=\"\" class=\"wp-image-59\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-9.png?w=789&amp;ssl=1 789w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-9.png?resize=300%2C236&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-9.png?resize=768%2C603&amp;ssl=1 768w\" sizes=\"auto, (max-width: 789px) 100vw, 789px\" \/><figcaption class=\"wp-element-caption\">Configuration de la source du Copy data<\/figcaption><\/figure>\n\n\n\n<p>Du c\u00f4t\u00e9 de la destination, nous devons maintenant pr\u00e9ciser de mani\u00e8re dynamique \u00e0 quel endroit d\u00e9poser notre fichier. Pour ce faire, nous avons les deux valeurs de notre table \u00ab\u00a0StagingDirectoryName\u00a0\u00bb pour le r\u00e9pertoire et \u00ab\u00a0FileName\u00a0\u00bb pour le nom de fichier.<\/p>\n\n\n\n<p>La subtilit\u00e9 \u00e0 ce moment est d&rsquo;avoir un dataset de destination param\u00e9trable. Pour ce faire, dans un dataset de fichier, il suffit de configurer le \u00ab\u00a0File path\u00a0\u00bb avec des param\u00e8tres et ce dataset deviendra \u00ab\u00a0g\u00e9n\u00e9rique\u00a0\u00bb pour l&rsquo;ensemble des fichiers que l&rsquo;on veut \u00e9crire dans notre lake.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"662\" height=\"383\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-14.png?resize=662%2C383&#038;ssl=1\" alt=\"\" class=\"wp-image-64\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-14.png?w=662&amp;ssl=1 662w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-14.png?resize=300%2C174&amp;ssl=1 300w\" sizes=\"auto, (max-width: 662px) 100vw, 662px\" \/><figcaption class=\"wp-element-caption\">Cr\u00e9ation des param\u00e8tres dans un dataset<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"386\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-15.png?resize=1024%2C386&#038;ssl=1\" alt=\"\" class=\"wp-image-65\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-15.png?resize=1024%2C386&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-15.png?resize=300%2C113&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-15.png?resize=768%2C289&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-15.png?w=1062&amp;ssl=1 1062w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><figcaption class=\"wp-element-caption\">Utilisation des param\u00e8tres pour le File path d&rsquo;un dataset<\/figcaption><\/figure>\n\n\n\n<p><em>(Mes exp\u00e9riences utilisent principalement des fichiers parquets et si vous vous demandez pourquoi je vous invite \u00e0 regarder cette petite vid\u00e9o : <a href=\"https:\/\/www.youtube.com\/watch?v=u00hCcpY6ng\">Synapse Espresso: CSV vs. Parquet? &#8211; YouTube<\/a>) <\/em><\/p>\n\n\n\n<p>Dans la destination du Copy data, il nous suffit maintenant d&rsquo;attribuer les valeurs de notre \u00ab\u00a0@item()\u00a0\u00bb (provenant du ForEach) aux param\u00e8tres de notre dataset g\u00e9n\u00e9rique :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"784\" height=\"745\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-13.png?resize=784%2C745&#038;ssl=1\" alt=\"\" class=\"wp-image-63\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-13.png?w=784&amp;ssl=1 784w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-13.png?resize=300%2C285&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-13.png?resize=768%2C730&amp;ssl=1 768w\" sizes=\"auto, (max-width: 784px) 100vw, 784px\" \/><figcaption class=\"wp-element-caption\">Configuration de la cible du Copy data<\/figcaption><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Premiere_execution\"><\/span>Premi\u00e8re ex\u00e9cution<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Notre pipeline est maintenant pr\u00eat. Une validation ou un publish est toujours de bon augure pour v\u00e9rifier que nous n&rsquo;avons rien oubli\u00e9 de critique.<\/p>\n\n\n\n<p>A ce stade, nous avons donc une seule ligne dans notre table de param\u00e9trage qui nous propose l&rsquo;extraction de la table \u00ab\u00a0Sales.Invoices\u00a0\u00bb \u00e0 aller d\u00e9poser dans le r\u00e9pertoire \u00ab\u00a0staging\/Invoices\/\u00a0\u00bb dans le fichier \u00ab\u00a0Invoices.parquet\u00a0\u00bb. Pour l&rsquo;exemple, je d\u00e9marrerais avec un datalake vide :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"420\" height=\"242\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-16.png?resize=420%2C242&#038;ssl=1\" alt=\"\" class=\"wp-image-67\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-16.png?w=420&amp;ssl=1 420w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-16.png?resize=300%2C173&amp;ssl=1 300w\" sizes=\"auto, (max-width: 420px) 100vw, 420px\" \/><figcaption class=\"wp-element-caption\">Illustration du datalake vide<\/figcaption><\/figure>\n\n\n\n<p>L&rsquo;execution du pipeline nous permet d&rsquo;observer l&rsquo;execution de l&rsquo;ensemble de nos activit\u00e9s :<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"581\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-25-1024x581.png?resize=1024%2C581\" alt=\"\" class=\"wp-image-92\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-25.png?resize=1024%2C581&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-25.png?resize=300%2C170&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-25.png?resize=768%2C436&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-25.png?w=1108&amp;ssl=1 1108w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><figcaption class=\"wp-element-caption\">Illustration de l&rsquo;execution du pipeline<\/figcaption><\/figure>\n\n\n\n<p>Et nous pouvons v\u00e9rifier que notre datalake contient maintenant nos donn\u00e9es dans un fichier parquet :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"298\" height=\"213\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-19.png?resize=298%2C213&#038;ssl=1\" alt=\"\" class=\"wp-image-70\"\/><figcaption class=\"wp-element-caption\">Illustration du datalake aliment\u00e9<\/figcaption><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Tout_ca_%E2%80%A6_pour_ca\"><\/span>Tout \u00e7a &#8230; pour \u00e7a ?<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Il est vrai que pour l&rsquo;instant, nous nous sommes un peu compliqu\u00e9 la vie et que pour ing\u00e9rer une table dans notre datalake on aurait pu utiliser directement un Copy data.<\/p>\n\n\n\n<p>Mais rappelons-nous du cas d&rsquo;usage ! nous avons une base de donn\u00e9es avec \u00ab\u00a0beaucoup\u00a0\u00bb de table, et nous voulons pouvoir en ing\u00e9rer que quelques-unes sans avoir rien \u00e0 toucher dans Synapse. <\/p>\n\n\n\n<p>C&rsquo;est maintenant que notre base de donn\u00e9es de param\u00e9trage va rentrer en action pour notre plus grand plaisir. Il me suffit d&rsquo;ex\u00e9cuter une requ\u00eate SQL (ou un script) pour changer le param\u00e9trage de mon pipeline et finalement ing\u00e9rer deux tables suppl\u00e9mentaires. Je vais donc ex\u00e9cuter les 2 requ\u00eates pr\u00e9c\u00e9demment en commentaire dans le script de d\u00e9marrage pour obtenir le r\u00e9sultat suivant :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"491\" height=\"218\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-20.png?resize=491%2C218&#038;ssl=1\" alt=\"\" class=\"wp-image-71\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-20.png?w=491&amp;ssl=1 491w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-20.png?resize=300%2C133&amp;ssl=1 300w\" sizes=\"auto, (max-width: 491px) 100vw, 491px\" \/><figcaption class=\"wp-element-caption\">Illustration des 3 lignes de param\u00e9trage dans notre table<\/figcaption><\/figure>\n\n\n\n<p>Sans aucune modification dans Synape, une nouvelle ex\u00e9cution du pipeline nous permet d&rsquo;ing\u00e9rer maintenant les trois tables pr\u00e9sentes dans la table de param\u00e9trage. On remarque que le Copy data est cette fois ex\u00e9cut\u00e9e trois fois :<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"689\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-26-1024x689.png?resize=1024%2C689\" alt=\"\" class=\"wp-image-93\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-26.png?resize=1024%2C689&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-26.png?resize=300%2C202&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-26.png?resize=768%2C517&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-26.png?w=1049&amp;ssl=1 1049w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><figcaption class=\"wp-element-caption\">Illustration de l&rsquo;execution du pipeline avec trois fois le Copy data<\/figcaption><\/figure>\n\n\n\n<p>Nous avons maintenant dans notre datalake les trois fichiers r\u00e9partis dans trois r\u00e9pertoires diff\u00e9rents :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"238\" height=\"256\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-22.png?resize=238%2C256&#038;ssl=1\" alt=\"\" class=\"wp-image-73\"\/><figcaption class=\"wp-element-caption\">Datalake aliment\u00e9 avec trois r\u00e9pertoires<\/figcaption><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Et_quoi_dautre\"><\/span>Et quoi d&rsquo;autre ?<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Cette construction nous a permis de param\u00e9trer notre pipeline et d&rsquo;ajouter facilement de nouvelles tables \u00e0 ing\u00e9rer tous en \u00e9tant capable de garder notre lake organis\u00e9. Nous n&rsquo;avons \u00e0 l&rsquo;instant que 3 param\u00e8tres dans notre table, mais en jouant un peu avec, on est capable de travailler sur d&rsquo;autres cas d&rsquo;usages. <\/p>\n\n\n\n<p>Envie d&rsquo;investiguer notre extraction sans \u00ab\u00a0casser\u00a0\u00bb nos fichiers existants ni notre arborescence ? On pourrait lancer une extraction dans un r\u00e9pertoire sp\u00e9cifique ? Pour ce faire, un simple update de notre table de param\u00e9trage est n\u00e9cessaire :<\/p>\n\n\n\n<p>UPDATE [param].[IngestionParam]<br>SET [StagingDirectoryName] = &lsquo;staging\/Investigation\/&rsquo;<\/p>\n\n\n\n<p><em>Oui, un UPDATE sans WHERE c&rsquo;est moche et ne doit probablement pas \u00eatre fait en production mais on est l\u00e0 pour l&rsquo;exp\u00e9rience !<\/em><\/p>\n\n\n\n<p>Nous sommes maintenant pr\u00eats \u00e0 investiguer :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"317\" height=\"158\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-23.png?resize=317%2C158&#038;ssl=1\" alt=\"\" class=\"wp-image-74\" srcset=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-23.png?w=317&amp;ssl=1 317w, https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-23.png?resize=300%2C150&amp;ssl=1 300w\" sizes=\"auto, (max-width: 317px) 100vw, 317px\" \/><figcaption class=\"wp-element-caption\">Illustration du datalake avec les fichiers g\u00e9n\u00e9r\u00e9s dans le r\u00e9pertoire \u00ab\u00a0Investigation\u00a0\u00bb<\/figcaption><\/figure>\n\n\n\n<p>sans avoir cass\u00e9 notre arborescence :<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"235\" height=\"177\" src=\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2022\/08\/image-24.png?resize=235%2C177&#038;ssl=1\" alt=\"\" class=\"wp-image-75\"\/><figcaption class=\"wp-element-caption\">Illustration du datalake avec la sauvegarde de l&rsquo;arborescence d&rsquo;origine<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Dans cet article nous avons vu comment param\u00e9trer et agir sur nos pipelines depuis l&rsquo;ext\u00e9rieur de Synapse. Cette m\u00e9thode n&rsquo;est pas standard et de ce fait, il est n\u00e9cessaire de s&rsquo;interroger consciencieusement pour savoir si elle r\u00e9pond \u00e0 votre besoin sans rajouter de complexit\u00e9 et de co\u00fbts superflus.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Pour_aller_plus_loin\"><\/span>Pour aller plus loin<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>L&rsquo;exp\u00e9rience pr\u00e9sent\u00e9e ici a \u00e9t\u00e9 d\u00e9finie pour pr\u00e9senter \u00ab\u00a0comment\u00a0\u00bb param\u00e9trer un pipeline depuis une base de donn\u00e9es. Cependant il est \u00e9videmment possible d&rsquo;aller encore plus loin dans la param\u00e9trisation de notre pipeline. Quelques exemples :<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Pouvoir extraire des donn\u00e9es depuis plusieurs bases diff\u00e9rentes h\u00e9berg\u00e9es sur plusieurs serveurs diff\u00e9rents<\/li>\n\n\n\n<li>Param\u00e9trer une extraction en \u00ab\u00a0Delta\u00a0\u00bb (et loguer en base les diff\u00e9rentes extractions et leurs param\u00e8tres)<\/li>\n\n\n\n<li>D\u00e9finir quelle table doit \u00eatre extraite \u00e0 quelle fr\u00e9quence<\/li>\n\n\n\n<li>Activer\/D\u00e9sactiver l&rsquo;extraction d&rsquo;une table par un Flag<\/li>\n\n\n\n<li>&#8230;<\/li>\n<\/ul>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Cet article \u00e9tait loin dans les cartons, mais l&rsquo;article Alternative pipeline parametrization for Azure Synapse Analytics a motiv\u00e9 la priorisation de cet article. Merci @paul_eng pour le partage et la motivation d&rsquo;\u00e9crire celui-ci !<\/p>\n","protected":false},"author":2,"featured_media":106,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[76],"tags":[10,9,8],"class_list":["post-45","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-synapsedatalake","tag-azure","tag-azure-synapse-analytics","tag-microsoft"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse ! - NiceData<\/title>\n<meta name=\"description\" content=\"Cet article \u00e9tait loin dans les cartons, mais l&#039;article Alternative pipeline parametrization for Azure Synapse Analytics a motiv\u00e9 la priorisation de cet article. Merci @paul_eng pour le partage et la motivation d&#039;\u00e9crire celui-ci !\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse ! - NiceData\" \/>\n<meta property=\"og:description\" content=\"Cet article \u00e9tait loin dans les cartons, mais l&#039;article Alternative pipeline parametrization for Azure Synapse Analytics a motiv\u00e9 la priorisation de cet article. Merci @paul_eng pour le partage et la motivation d&#039;\u00e9crire celui-ci !\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/\" \/>\n<meta property=\"og:site_name\" content=\"NiceData\" \/>\n<meta property=\"article:published_time\" content=\"2022-08-19T06:40:02+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-10-01T13:04:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1920\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Jean-Laurent Ferralis\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@JLFerralis\" \/>\n<meta name=\"twitter:site\" content=\"@JLFerralis\" \/>\n<meta name=\"twitter:label1\" content=\"\u00c9crit par\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jean-Laurent Ferralis\" \/>\n\t<meta name=\"twitter:label2\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data2\" content=\"15 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"TechArticle\",\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/\"},\"author\":{\"name\":\"Jean-Laurent Ferralis\",\"@id\":\"https:\/\/www.nicedata.fr\/#\/schema\/person\/8d1ad38004d3b0cf6bff7c200c795e19\"},\"headline\":\"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse !\",\"datePublished\":\"2022-08-19T06:40:02+00:00\",\"dateModified\":\"2024-10-01T13:04:28+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/\"},\"wordCount\":2219,\"commentCount\":2,\"publisher\":{\"@id\":\"https:\/\/www.nicedata.fr\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920\",\"keywords\":[\"Azure\",\"Azure Synapse Analytics\",\"Microsoft\"],\"articleSection\":[\"Azure Synapse - Datalake\"],\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/\",\"url\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/\",\"name\":\"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse ! - NiceData\",\"isPartOf\":{\"@id\":\"https:\/\/www.nicedata.fr\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920\",\"datePublished\":\"2022-08-19T06:40:02+00:00\",\"dateModified\":\"2024-10-01T13:04:28+00:00\",\"description\":\"Cet article \u00e9tait loin dans les cartons, mais l'article Alternative pipeline parametrization for Azure Synapse Analytics a motiv\u00e9 la priorisation de cet article. Merci @paul_eng pour le partage et la motivation d'\u00e9crire celui-ci !\",\"breadcrumb\":{\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#breadcrumb\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage\",\"url\":\"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920\",\"contentUrl\":\"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920\",\"width\":2560,\"height\":1920,\"caption\":\"Photo par the blowup sur Unsplash\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.nicedata.fr\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse !\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.nicedata.fr\/#website\",\"url\":\"https:\/\/www.nicedata.fr\/\",\"name\":\"NiceData\",\"description\":\"L&#039;expertise Data du sud\",\"publisher\":{\"@id\":\"https:\/\/www.nicedata.fr\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.nicedata.fr\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.nicedata.fr\/#organization\",\"name\":\"NiceData\",\"url\":\"https:\/\/www.nicedata.fr\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/www.nicedata.fr\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2024\/11\/NICE-DATA_JLFMod.webp?fit=2493%2C1249&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2024\/11\/NICE-DATA_JLFMod.webp?fit=2493%2C1249&ssl=1\",\"width\":2493,\"height\":1249,\"caption\":\"NiceData\"},\"image\":{\"@id\":\"https:\/\/www.nicedata.fr\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/JLFerralis\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.nicedata.fr\/#\/schema\/person\/8d1ad38004d3b0cf6bff7c200c795e19\",\"name\":\"Jean-Laurent Ferralis\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/3cbfdae273d44fb82b902a3451eb0db37485119e34cb70c1fdd186c9b731b9f3?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/3cbfdae273d44fb82b902a3451eb0db37485119e34cb70c1fdd186c9b731b9f3?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/3cbfdae273d44fb82b902a3451eb0db37485119e34cb70c1fdd186c9b731b9f3?s=96&d=mm&r=g\",\"caption\":\"Jean-Laurent Ferralis\"},\"description\":\"French Data Professionnal - BI consultant and #sql lover. I also #swimbikerun when possible ! Living in @villedenice\",\"sameAs\":[\"http:\/\/xp-it.com\"],\"url\":\"https:\/\/www.nicedata.fr\/index.php\/author\/jlf\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse ! - NiceData","description":"Cet article \u00e9tait loin dans les cartons, mais l'article Alternative pipeline parametrization for Azure Synapse Analytics a motiv\u00e9 la priorisation de cet article. Merci @paul_eng pour le partage et la motivation d'\u00e9crire celui-ci !","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/","og_locale":"fr_FR","og_type":"article","og_title":"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse ! - NiceData","og_description":"Cet article \u00e9tait loin dans les cartons, mais l'article Alternative pipeline parametrization for Azure Synapse Analytics a motiv\u00e9 la priorisation de cet article. Merci @paul_eng pour le partage et la motivation d'\u00e9crire celui-ci !","og_url":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/","og_site_name":"NiceData","article_published_time":"2022-08-19T06:40:02+00:00","article_modified_time":"2024-10-01T13:04:28+00:00","og_image":[{"height":1920,"width":2560,"url":"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920","type":"image\/jpeg"}],"author":"Jean-Laurent Ferralis","twitter_card":"summary_large_image","twitter_creator":"@JLFerralis","twitter_site":"@JLFerralis","twitter_misc":{"\u00c9crit par":"Jean-Laurent Ferralis","Dur\u00e9e de lecture estim\u00e9e":"15 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"TechArticle","@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#article","isPartOf":{"@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/"},"author":{"name":"Jean-Laurent Ferralis","@id":"https:\/\/www.nicedata.fr\/#\/schema\/person\/8d1ad38004d3b0cf6bff7c200c795e19"},"headline":"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse !","datePublished":"2022-08-19T06:40:02+00:00","dateModified":"2024-10-01T13:04:28+00:00","mainEntityOfPage":{"@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/"},"wordCount":2219,"commentCount":2,"publisher":{"@id":"https:\/\/www.nicedata.fr\/#organization"},"image":{"@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage"},"thumbnailUrl":"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920","keywords":["Azure","Azure Synapse Analytics","Microsoft"],"articleSection":["Azure Synapse - Datalake"],"inLanguage":"fr-FR","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/","url":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/","name":"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse ! - NiceData","isPartOf":{"@id":"https:\/\/www.nicedata.fr\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage"},"image":{"@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage"},"thumbnailUrl":"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920","datePublished":"2022-08-19T06:40:02+00:00","dateModified":"2024-10-01T13:04:28+00:00","description":"Cet article \u00e9tait loin dans les cartons, mais l'article Alternative pipeline parametrization for Azure Synapse Analytics a motiv\u00e9 la priorisation de cet article. Merci @paul_eng pour le partage et la motivation d'\u00e9crire celui-ci !","breadcrumb":{"@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#breadcrumb"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/"]}]},{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#primaryimage","url":"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920","contentUrl":"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920","width":2560,"height":1920,"caption":"Photo par the blowup sur Unsplash"},{"@type":"BreadcrumbList","@id":"https:\/\/www.nicedata.fr\/index.php\/2022\/08\/19\/parametrer-les-pipeline-azure-synapse-analytics-sans-ouvrir-synapse\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.nicedata.fr\/"},{"@type":"ListItem","position":2,"name":"Param\u00e9trer les Pipeline Azure Synapse Analytics sans ouvrir Synapse !"}]},{"@type":"WebSite","@id":"https:\/\/www.nicedata.fr\/#website","url":"https:\/\/www.nicedata.fr\/","name":"NiceData","description":"L&#039;expertise Data du sud","publisher":{"@id":"https:\/\/www.nicedata.fr\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.nicedata.fr\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"fr-FR"},{"@type":"Organization","@id":"https:\/\/www.nicedata.fr\/#organization","name":"NiceData","url":"https:\/\/www.nicedata.fr\/","logo":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/www.nicedata.fr\/#\/schema\/logo\/image\/","url":"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2024\/11\/NICE-DATA_JLFMod.webp?fit=2493%2C1249&ssl=1","contentUrl":"https:\/\/i0.wp.com\/www.nicedata.fr\/wp-content\/uploads\/2024\/11\/NICE-DATA_JLFMod.webp?fit=2493%2C1249&ssl=1","width":2493,"height":1249,"caption":"NiceData"},"image":{"@id":"https:\/\/www.nicedata.fr\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/JLFerralis"]},{"@type":"Person","@id":"https:\/\/www.nicedata.fr\/#\/schema\/person\/8d1ad38004d3b0cf6bff7c200c795e19","name":"Jean-Laurent Ferralis","image":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/secure.gravatar.com\/avatar\/3cbfdae273d44fb82b902a3451eb0db37485119e34cb70c1fdd186c9b731b9f3?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/3cbfdae273d44fb82b902a3451eb0db37485119e34cb70c1fdd186c9b731b9f3?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/3cbfdae273d44fb82b902a3451eb0db37485119e34cb70c1fdd186c9b731b9f3?s=96&d=mm&r=g","caption":"Jean-Laurent Ferralis"},"description":"French Data Professionnal - BI consultant and #sql lover. I also #swimbikerun when possible ! Living in @villedenice","sameAs":["http:\/\/xp-it.com"],"url":"https:\/\/www.nicedata.fr\/index.php\/author\/jlf\/"}]}},"jetpack_featured_media_url":"https:\/\/images.unsplash.com\/photo-1622534376374-fe4480328daa?ixid=MnwzNTY4Mjl8MHwxfGFsbHx8fHx8fHx8fDE2NjA4OTExODg&ixlib=rb-1.2.1&fm=jpg&q=85&fit=crop&w=2560&h=1920","jetpack-related-posts":[],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/posts\/45","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/comments?post=45"}],"version-history":[{"count":26,"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/posts\/45\/revisions"}],"predecessor-version":[{"id":675,"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/posts\/45\/revisions\/675"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/media\/106"}],"wp:attachment":[{"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/media?parent=45"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/categories?post=45"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nicedata.fr\/index.php\/wp-json\/wp\/v2\/tags?post=45"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}