Beam学习笔记3：Flink Streaming Pipeline Translator

FlinkStreamingPipelineTranslator

1 所有的可用Translator：

// here you can find all the available translators.
  static {
    TRANSLATORS.put(Read.Bounded.class,new BoundedReadSourceTranslator());
    TRANSLATORS.put(Read.Unbounded.class,new UnboundedReadSourceTranslator());
    TRANSLATORS.put(TextIO.Write.Bound.class,new TextIOWriteBoundStreamingTranslator());

    TRANSLATORS.put(ParDo.MultIoUtput.class,new ParDoStreamingTranslator());
    TRANSLATORS.put(
        SplittableParDo.ProcessElements.class,new SplittableProcessElementsstreamingTranslator());
    TRANSLATORS.put(
        SplittableParDo.GBKIntoKeyedWorkItems.class,new GBKIntoKeyedWorkItemsTranslator());


    TRANSLATORS.put(Window.Assign.class,new WindowAssignTranslator());
    TRANSLATORS.put(Flatten.PCollections.class,new FlattenPCollectionTranslator());
    TRANSLATORS.put(
        FlinkStreamingViewOverrides.CreateFlinkPCollectionView.class,new CreateViewStreamingTranslator());

    TRANSLATORS.put(Reshuffle.class,new ReshuffleTranslatorStreaming());
    TRANSLATORS.put(GroupByKey.class,new GroupByKeyTranslator());
    TRANSLATORS.put(Combine.PerKey.class,new CombinePerKeyTranslator());
  }

2 applyStreamingTransform：

基本逻辑：将PTransform关联到streamingContext

// create the applied PTransform on the streamingContext streamingContext.setCurrentTransform(node.toAppliedPTransform());
    typedTranslator.translateNode(typedTransform,streamingContext);

12种类型的translator多态实现了StreamTransformTranslator.translateNode(T transform,FlinkStreamingTranslationContext context);方法。

原文地址：https://www.jb51.cc/javaschema/283146.html