Previous | Next --- Slide 32 of 55
Back to Lecture Thumbnails
It's me!

Lineage is sequence of transformations which convert data in the persistent memory into the required result by creating multiple RDDs on its way.

awu

How expensive is it to create and maintain these lineages?

noelma

Spark's fault tolerance can be attributed to its use of RDDs? As professor Kunle put it, a lineage of RDDs let's us recreate a sequence of transformations to recover some intermediate data. Where are these RDDs stored/saved relative to nodes?

jkuro

@awu I think it depends on the implementation of the RDDs. RDDs are just an abstraction

A bar cat

I still do not completely understand what an RDD is. To check my understanding, an RDD is its own data structure that cannot be changed?

Please log in to leave a comment.