Databricks Certified Associate Developer for Apache Spark 3.5 - Python (Associate-Developer-Apache-Spark-3.5) Free Practice Test

Question 1

A data engineer uses a broadcast variable to share a DataFrame containing millions of rows across executors for lookup purposes. What will be the outcome?

A. The job will hang indefinitely as Spark will struggle to distribute and serialize such a large broadcast variable to all executors

B. The job may fail if the executors do not have enough CPU cores to process the broadcasted dataset

C. The job may fail if the memory on each executor is not large enough to accommodate the DataFrame being broadcasted

D. The job may fail because the driver does not have enough CPU cores to serialize the large DataFrame

Correct Answer: C

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 2

49 of 55.
In the code block below, aggDF contains aggregations on a streaming DataFrame:
aggDF.writeStream \
.format("console") \
.outputMode("???") \
.start()
Which output mode at line 3 ensures that the entire result table is written to the console during each trigger execution?

A. COMPLETE

B. APPEND

C. REPLACE

D. AGGREGATE

Correct Answer: A

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 3

A data engineer noticed improved performance after upgrading from Spark 3.0 to Spark 3.5. The engineer found that Adaptive Query Execution (AQE) was enabled.
Which operation is AQE implementing to improve performance?

A. Dynamically switching join strategies

B. Optimizing the layout of Delta files on disk

C. Collecting persistent table statistics and storing them in the metastore for future use

D. Improving the performance of single-stage Spark jobs

Correct Answer: A

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 4

Given:
python
CopyEdit
spark.sparkContext.setLogLevel("<LOG_LEVEL>")
Which set contains the suitable configuration settings for Spark driver LOG_LEVELs?

A. ALL, DEBUG, FAIL, INFO

B. WARN, NONE, ERROR, FATAL

C. FATAL, NONE, INFO, DEBUG

D. ERROR, WARN, TRACE, OFF

Correct Answer: D

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 5

An engineer has a large ORC file located at /file/test_data.orc and wants to read only specific columns to reduce memory usage.
Which code fragment will select the columns, i.e., col1, col2, during the reading process?

A. spark.read.format("orc").select("col1", "col2").load("/file/test_data.orc")

B. spark.read.orc("/file/test_data.orc").filter("col1 = 'value' ").select("col2")

C. spark.read.format("orc").load("/file/test_data.orc").select("col1", "col2")

D. spark.read.orc("/file/test_data.orc").selected("col1", "col2")

Correct Answer: C

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 6

A data engineer is reviewing a Spark application that applies several transformations to a DataFrame but notices that the job does not start executing immediately.
Which two characteristics of Apache Spark's execution model explain this behavior?
Choose 2 answers:

A. Transformations are executed immediately to build the lineage graph.

B. Only actions trigger the execution of the transformation pipeline.

C. The Spark engine requires manual intervention to start executing transformations.

D. The Spark engine optimizes the execution plan during the transformations, causing delays.

E. Transformations are evaluated lazily.

Correct Answer: B,E

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 7

48 of 55.
A data engineer needs to join multiple DataFrames and has written the following code:
from pyspark.sql.functions import broadcast
data1 = [(1, "A"), (2, "B")]
data2 = [(1, "X"), (2, "Y")]
data3 = [(1, "M"), (2, "N")]
df1 = spark.createDataFrame(data1, ["id", "val1"])
df2 = spark.createDataFrame(data2, ["id", "val2"])
df3 = spark.createDataFrame(data3, ["id", "val3"])
df_joined = df1.join(broadcast(df2), "id", "inner") \
.join(broadcast(df3), "id", "inner")
What will be the output of this code?

A. The code will fail because the second join condition (df2.id == df3.id) is incorrect.

B. The code will result in an error because broadcast() must be called before the joins, not inline.

C. The code will fail because only one broadcast join can be performed at a time.

D. The code will work correctly and perform two broadcast joins simultaneously to join df1 with df2, and then the result with df3.

Correct Answer: D

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 8

46 of 55.
A data engineer is implementing a streaming pipeline with watermarking to handle late-arriving records.
The engineer has written the following code:
inputStream \
.withWatermark("event_time", "10 minutes") \
.groupBy(window("event_time", "15 minutes"))
What happens to data that arrives after the watermark threshold?

A. Any data arriving more than 10 minutes after the watermark threshold will be ignored and not included in the aggregation.

B. Records that arrive later than the watermark threshold (10 minutes) will automatically be included in the aggregation if they fall within the 15-minute window.

C. Data arriving more than 10 minutes after the latest watermark will still be included in the aggregation but will be placed into the next window.

D. The watermark ensures that late data arriving within 10 minutes of the latest event time will be processed and included in the windowed aggregation.

Correct Answer: A

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 9

A data engineer is building an Apache Spark™ Structured Streaming application to process a stream of JSON events in real time. The engineer wants the application to be fault-tolerant and resume processing from the last successfully processed record in case of a failure. To achieve this, the data engineer decides to implement checkpoints.
Which code snippet should the data engineer use?

A. query = streaming_df.writeStream \
.format("console") \
.outputMode("complete") \
.start()

B. query = streaming_df.writeStream \
.format("console") \
.outputMode("append") \
.start()

C. query = streaming_df.writeStream \
.format("console") \
.outputMode("append") \
.option("checkpointLocation", "/path/to/checkpoint") \
.start()

D. query = streaming_df.writeStream \
.format("console") \
.option("checkpoint", "/path/to/checkpoint") \
.outputMode("append") \
.start()

Correct Answer: C

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Welcome to TestSimulate

Databricks Certified Associate Developer for Apache Spark 3.5 - Python (Associate-Developer-Apache-Spark-3.5) Free Practice Test