Add background scheduler

#1
Data Is Better Together org
Data Is Better Together org

Hi @ZennyKenny ! Sorry for the silence on my side, we got some bank holidays in Spain. I'ma try it out on a separate dashboard to see if nothing breaks, and if so, we can push it to the template and spread the word for people to introduce it the dashboard of the different languages.

Data Is Better Together org

Please note that this should be merged with this PR as well to include required packages.

Data Is Better Together org

Hi again @ZennyKenny , I've been trying out your proposals here https://huggingface.co./spaces/ignacioct/TryingRestartDashboard, using the Spanish Dashboard to fetch the data. I've added the parameter factory_reboot=True in restart_space(), and the plots are updating just fine, but I've been able to get an update on the leaderboard dataframe, unless I manually go to the Space options in the web UI and trigger a factory reboot. Were you able to see updates on the Dataframe? I'm starting to thing that the Dataframe caches between restarts, but I do not understand why I'm apparently not getting factory reboots.

Data Is Better Together org
โ€ข
edited Apr 3

Hm, the only clue I could find at first glance is Line 49:

    restart_space(
        "DIBT/TemplateDashboardPromptTranslation",
        token=os.getenv("HF_TOKEN"),
        # factory_reboot=True,
    )

We declare "DIBT/TemplateDashboardPromptTranslation" but target ought to be "ignacioct/TryingRestartDashboard" if I am not mistaken.

Maybe you use a write key from Spanish-language dashboard (?) (no visibility here of course), but it looks like the goal here is to reset dashboard at ignacioct/TryingRestartDashboard.

Data Is Better Together org

Thanks for spotting that one out! It was definitely a dumb mistake on my side. I've been able to get real updates, but only if factory_reboot is settled. Normal resets apparently do not update the DataFrame. We were wondering, isn't it too heavy on the system to do a reboot each 30 mins? It takes about five mins to be back up, with the updated info.

Data Is Better Together org
โ€ข
edited Apr 3

Hm, I wasn't using factory_reboot at all. Please let me have a closer look and see if I can find something relevant.

Do you have any log output that might be helpful? I have some hypotheses but I am not sure that they are relevant.

Data Is Better Together org

@ZennyKenny @ignacioct , can I just check if we still need to merge this or if this is superseded by https://github.com/huggingface/data-is-better-together/pull/46/files

Data Is Better Together org

Closing

ZennyKenny changed pull request status to closed

Sign up or log in to comment