Prepare with Databricks : Databricks-Certified-Data-Engineer-Professional exam braindumps as your best preparation materials

A data engineer, while designing a Pandas UDF to process financial time-series data with complex calculations that require maintaining state across rows within each stock symbol group, must ensure the function is efficient and scalable. Which approach will solve the problem with minimum overhead while preserving data integrity?

• A. Use a SCALAR Pandas UDF that processes the entire dataset at once, implementing custom partitioning logic within the UDF to group by stock symbol and maintain state using global variables shared across all executor processes.
• B. Use a SCALAR_ITER Pandas UDF with iterator-based processing, implementing state management through persistent storage (Delta tables) that gets updated after each batch to maintain continuity across iterator chunks.
• C. Use a grouped_agg Pandas UDF that processes each stock symbol group independently, maintaining state through intermediate aggregation results that get passed between successive UDF calls via broadcast variables.
• D. Use applyInPandas() on a Spark DataFrame that receives all rows for each stock symbol as a Pandas DataFrame, allowing processing within each group while maintaining state variables local to each group's processing function.

The data governance team has instituted a requirement that all tables containing Personal Identifiable Information (PH) must be clearly annotated. This includes adding column comments, table comments, and setting the custom table property "contains_pii" = true.
The following SQL DDL statement is executed to create a new table:

Which command allows manual confirmation that these three requirements have been met?

• A. DESCRIBE HISTORY dev.pii test
• B. DESCRIBE DETAIL dev.pii test
• C. SHOW TBLPROPERTIES dev.pii test
• D. SHOW TABLES dev
• E. DESCRIBE EXTENDED dev.pii test

Which statement describes integration testing?

• A. Validates interactions between subsystems of your application
• B. Requires an automated testing framework
• C. Validates behavior of individual elements of your application
• D. Requires manual intervention
• E. Validates an application use case

A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Events are recorded once per minute per device.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:

Choose the response that correctly fills in the blank within the code block to complete this task.

• A. window("event_time", "10 minutes").alias("time")
• B. "event_time"
• C. window("event_time", "5 minutes").alias("time")
• D. to_interval("event_time", "5 minutes").alias("time")
• E. lag("event_time", "10 minutes").alias("time")

A platform engineer is creating catalogs and schemas for the development team to use.
The engineer has created an initial catalog, catalog_A, and initial schema, schema_A. The engineer has also granted USE CATALOG, USE SCHEMA, and CREATE TABLE to the development team so that the engineer can begin populating the schema with new tables.
Despite being owner of the catalog and schema, the engineer noticed that they do not have access to the underlying tables in Schema_A.
What explains the engineer's lack of access to the underlying tables?

• A. The platform engineer needs to execute a REFRESH statement as the table permissions did not automatically update for owners.
• B. Users granted with USE CATALOG can modify the owner's permissions to downstream tables.
• C. Permissions explicitly given by the table creator are the only way the Platform Engineer could access the underlying tables in their schema.
• D. The owner of the schema does not automatically have permission to tables within the schema, but can grant them to themselves at any point.