Pass with professional Databricks-Certified-Data-Engineer-Professional actual quiz materials

What is a method of installing a Python package scoped at the notebook level to all nodes in the currently active cluster?

• A. Use &Pip install in a notebook cell
• B. Install libraries from PyPi using the cluster UI
• C. Use &sh install in a notebook cell
• D. Run source env/bin/activate in a notebook setup script

Explanation: Only visible for ActualTestsQuiz members. You can sign-up / login (it's free).

An upstream system has been configured to pass the date for a given batch of data to the Databricks Jobs API as a parameter. The notebook to be scheduled will use this parameter to load data with the following code:
df = spark.read.format("parquet").load(f"/mnt/source/(date)")
Which code block should be used to create the date Python variable used in the above code block?

• A. date = dbutils.notebooks.getParam("date")
• B. import sys
  date = sys.argv[1]
• C. dbutils.widgets.text("date", "null")
  date = dbutils.widgets.get("date")
• D. date = spark.conf.get("date")
• E. input_dict = input()
  date= input_dict["date"]

Explanation: Only visible for ActualTestsQuiz members. You can sign-up / login (it's free).

A Delta Lake table was created with the below query:

Realizing that the original query had a typographical error, the below code was executed:
ALTER TABLE prod.sales_by_stor RENAME TO prod.sales_by_store
Which result will occur after running the second command?

• A. A new Delta transaction log Is created for the renamed table.
• B. All related files and metadata are dropped and recreated in a single ACID transaction.
• C. The table reference in the metastore is updated and all data files are moved.
• D. The table reference in the metastore is updated and no data is changed.
• E. The table name change is recorded in the Delta transaction log.

Explanation: Only visible for ActualTestsQuiz members. You can sign-up / login (it's free).

A data engineer is designing a system to process batch patient encounter data stored in an S3 bucket, creating a Delta table (patient_encounters) with columns encounter_id, patient_id, encounter_date, diagnosis_code, and treatment_cost. The table is queried frequently by patient_id and encounter_date, requiring fast performance. Fine-grained access controls must be enforced. The engineer wants to minimize maintenance and boost performance. How should the data engineer create the patient_encounters table?

• A. Create an external table in Unity Catalog, specifying an S3 location for the data files. Enable predictive optimization through table properties, and configure Unity Catalog permissions for access controls.
• B. Create a managed table in Hive Metastore. Configure Hive Metastore permissions for access controls, and rely on predictive optimization to enhance query performance and simplify maintenance.
• C. Create a managed table in Unity Catalog. Configure Unity Catalog permissions for access controls, schedule jobs to run OPTIMIZE and VACUUM commands daily to achieve best performance.
• D. Create a managed table in Unity Catalog. Configure Unity Catalog permissions for access controls, and rely on predictive optimization to enhance query performance and simplify maintenance.

Explanation: Only visible for ActualTestsQuiz members. You can sign-up / login (it's free).

A nightly batch job is configured to ingest all data files from a cloud object storage container where records are stored in a nested directory structure YYYY/MM/DD. The data for each date represents all records that were processed by the source system on that date, noting that some records may be delayed as they await moderator approval. Each entry represents a user review of a product and has the following schema:
user_id STRING, review_id BIGINT, product_id BIGINT, review_timestamp TIMESTAMP, review_text STRING The ingestion job is configured to append all data for the previous date to a target table reviews_raw with an identical schema to the source system. The next step in the pipeline is a batch write to propagate all new records inserted into reviews_raw to a table where data is fully deduplicated, validated, and enriched.
Which solution minimizes the compute costs to propagate this batch of data?

• A. Filter all records in the reviews_raw table based on the review_timestamp; batch append those records produced in the last 48 hours.
• B. Configure a Structured Streaming read against the reviews_raw table using the trigger once execution mode to process new records as a batch job.
• C. Use Delta Lake version history to get the difference between the latest version of reviews_raw and one version prior, then write these records to the next table.
• D. Reprocess all records in reviews_raw and overwrite the next table in the pipeline.
• E. Perform a batch read on the reviews_raw table and perform an insert-only merge using the natural composite key user_id, review_id, product_id, review_timestamp.

Explanation: Only visible for ActualTestsQuiz members. You can sign-up / login (it's free).