screen shot showing replay info instead of the actual gameplay

[HadiSDev]

If you encountered an issue, please post the following things:

• description of an issue:
  I have 50 saved replays from the basic scenario, that I looping over and playing. For every action I save the screen shot image however many of the images are images of the replay info like
  
  And
  
  
  For some episodes all the screen shots are from the replay info. It worked fine on Linux but on windows it is not working.

Also I did notice that replays do contain all the images from when it was save. Is that normal?

• ViZDoom version you are using,
  1.1.7
• information about your OS platform, distribution and environment (GCC, Python, Java, Lua versions etc.),
  Windows 10 Pro x64 Python 3.7

Thanks!

[Miffyli]

I am not sure what the issue is here? I understand you are recording your own replay by playing the game and saving the images with your own script? Or are you using the replay files of ViZDoom to record things?

In any case, you seem to be launching the game outside vizdoom control. I recommend you launch the game and control the flow via ViZDoom. See this example for recording Doom replay files, for example.

[HadiSDev]

@Miffyli Yea sorry I forgot to mention I am using the vizdoom replay files here:
Replay.zip

I have also followed the examples linked to record and to replay

[Miffyli]

Ok, I ran the replays on Windows 10, ViZDoom 1.1.7, Python 3.7. If I use SPECTATOR mode, it will have "Main menu" overlay on top and also include some of the console frames in state.screen_buffer. If I use ASYNC_PLAYER mode, there is no "Main menu" overlay but it still includes some images of console. It seems like the first (or last?) frames of replays are these console images.

That said, for me things seem to work as expected, minus the occasional image of the console. Here's the code I used.

from vizdoom import *
import cv2

# Run with basic.wad and basic.cfg in same directory,
# and replays in "replays" directory
game = DoomGame()
game.load_config("basic.cfg")

game.set_screen_resolution(ScreenResolution.RES_800X600)
game.set_render_hud(True)
game.set_mode(Mode.ASYNC_PLAYER)
game.init()

episodes = 5

for i in range(episodes):
    game.replay_episode("replays/episode_basic_" + str(i) + "_rec.lmp")

    while not game.is_episode_finished():
        s = game.get_state()
        buf = s.screen_buffer
        buf = buf.transpose([1,2,0])
        cv2.imshow("img", buf)
        cv2.waitKey(1)

        # Use advance_action instead of make_action.
        game.advance_action()

        r = game.get_last_reward()
        # game.get_last_action is not supported and don't work for replay at the moment.

        print("State #" + str(s.number))
        print("Game variables:", s.game_variables[0])
        print("Reward:", r)
        print("=====================")

    print("Episode finished.")
    print("total reward:", game.get_total_reward())
    print("************************")

[HadiSDev]

Okay thanks I will try again. Btw game.get_last_action does work for some actions

[HadiSDev]

Oh I noticed you only ran 5 episodes. As the number of episodes you are replaying is increasing the ratio of console images increases as well. Try to use cv2 to save all the images in a folder for all episodes I attached

[Miffyli]

Ah yes, now I see the problem. Indeed if I run all 50, around 35th replay it starts to get stuck in the console screen and behave oddly. If I only play last 20 (replays 30-50), it still works as expected, indicating there is some kind of issue with loading multiple replays in row. Thanks for reporting this!

Can you confirm this does not happen on Linux (I do not have access to a Linux machine at the moment)? All development is done on Linux side, so Windows side has odd bugs like this every now and then, which is why we highly recommend using Linux machines.

As for the get_last_action: This is a known issue (e.g. #354), so indeed you need to be careful with the actions you get back.

[HadiSDev]

@Miffyli I also noticed that the amount of screen shots in a replay file is less than the actual amount from when it was recorded. Is that normal?

[Miffyli]

Is this with all replay files or just after loading many replays (like discussed above)? How many frames were missing? Can you test this by recording and replaying them on Linux? When I was playing your replays, they seemed fine and contain necessary frames.

[HadiSDev]

For example in the home scenarios I have 99 episodes. When I recorded them I also saved every image (13K) images. When I run the recorded files, I get 4.2K images. This is the same on windows and linux

[Miffyli]

Hmm I have not observed such behaviour before. Have you noticed what is missing, e.g. does it cut off from the beginning or from the end of the episodes, or does it skip every Nth frame?

@mwydmuch Would you have an insight on this?

[mwydmuch]

Hi @HadiSDev, thank you for reporting the problem! I used recordings multiple times and I've never encountered such issues so I don't have any insights on this. I work almost only on Linux/macOS, so it probably occurs only on Windows. I will look into this, but I don't have Windows at the moment so it will take me a few days.

[Miffyli]

@mwydmuch Cheers! Judging by above comments it also happens on Linux. If you need help with Windows debugging, nudge me :).

[HadiSDev]

Steps to reproduce:

1. Play fair amount of episodes on any scenario. (I tried on MyWayHome with 99 episodes). You have to save all screen buffers in a folder.
2. Count all files in the image folder that you played
3. Now run all the replays again and save the screen buffers in a new image folder
4. Count all the files again
5. Compare