vb.net screenshot tutorial

[abuckau907_num2]

The Goal
Basically, take screenshots of a specific app (game?) and 'scan' (ie. analyze) the image for certain things (ie. colors: health bar, ememies, etc).

Basic Assumptions
You already know how to program in vb.net. You know variables, functions, classes, ByRef: most of the basics. I'm not going to say "Create new project" or specifically say where to declare variables.


Getting Started
(I lied, make a new project )

1) How exactly do we take the screenshot?
2) How do we scan over (analyze) that screenshot?
3) What is a screenshot?

..I'm answering in reverse order, but...

3) Since we are using Microsoft's .Net Framework for our project, we're going to use a few .net classes/functions. Microsoft likes 'bitmaps'(reference needed?), so if we want to leverage existing .net code, there are a few things we 'have' to do. Back to the point, our screenshot will be stored as a 'Bitmap' object. (probably System.Drawing.Bitmap? I don't have msvs open atm. google tells)
2) The bitmap class is pretty complex, and I'm not a graphics expert. The class has a function .GetPixel(x,y) which will return a 'color' object that we want. X,Y being the location in the bitmap of course. MAJOR point: It would be faster! to store the data in an array, and access it that way, instead of calling a .Function() for each and every pixel in the data! But..it works, and it's not much code. There will be a part 2 to this tutorial eventually. anyway.
1) The .net framework of course. There is a class called 'Graphics' and a method called 'CopyFromScreen'. <--Screen = your physical monitor, ie. what you're looking at ie. screenshot area

So, on to some code.
I created a class called "GooeyManager" (play on words), but feel free to add it wherever you want. All of these functions can/should be made static, ie. you don't need an object of the guimgr class before you can access the functions. Change it if you want to.

Just a demo

Dim _myBitmap as new Bitmap(50,50) '' 50,50 is the size: width, height, few diff. overloads
Dim _gr As Graphics = Graphics.FromImage(_myBitmap) '' we need the gr object for copyfromscreen

_gr.CopyFromScreen(***,***) '' not actual paremeters.
_gr.Dispose() ''don't forget about this later

_myBitmap : object of the Bitmap class. Stores the ss (screenshot) for us.
_gr : Graphics class. It's a little complex. CopyFromScreen() is pretty straight forward.
.CopyFromScreen() : ie. Take screenshot (of specific region on the monitor)

[some real code]

    Public Shared Function CopyFromScreen(ByVal tl As Point, ByVal width As Int32, ByVal height As Int32) As Bitmap
        Dim _rtnBmp As New Bitmap(width, height)
        Dim _gr As Graphics = Graphics.FromImage(_rtnBmp)
        _gr.CopyFromScreen(tl, New Point(0, 0), New Size(width, height), CopyPixelOperation.SourceCopy) '' sourcecopy or ____ ?
        _gr.Dispose()
        Return _rtnBmp
    End Function

description of parameters/arguments
tl : top-left, the top-left {x,y} of the region you want to copy
br : bottom-right, the bottom-right {x,y} of the region you want to copy

new Point(0,0) and New Size(*,*) : basically CopyFromScreen() doesn't have to use our entire bitmap.


That's it!
Kind of short, not a lot of code, but it can sometimes be useful.

Just a little more, but you might not need this.
take a ss of a specific form
Basically it users a few more Windows API to find a specific window (app,game,whatever) and take a screenshot of only that window (assuming it's open and top-most..)

    Public Shared Function GetWindowScreen(ByVal windowTitle As String, Optional ByVal includeBorders As Boolean = True) As Bitmap
        Dim _targetWindowHandle As IntPtr = FindWindow(Nothing, windowTitle) 'try to find window based on title.
        If _targetWindowHandle = 0 Then
            ''window not found
            MessageBox.Show("ScreenShotManager::GetScreenOfApp windowTitle not found! --> '" & windowTitle & "'" & Environment.NewLine _
                            & "Are you sure that program is running?")
            Return New Bitmap(2, 2) ''why not (1,1). choice.
        Else
            ''window found
            Dim _targetWindowInfo As WINDOWINFO
            GetWindowInfo(_targetWindowHandle, _targetWindowInfo)
            If includeBorders Then
                Dim _tl As New Point(_targetWindowInfo.rcWindow.Left, _targetWindowInfo.rcWindow.Top)
                Return CopyFromScreen(_tl, _targetWindowInfo.rcWindow.Width, _targetWindowInfo.rcWindow.Height)
            Else
                Dim _tl As New Point(_targetWindowInfo.rcClient.Left, _targetWindowInfo.rcClient.Top)
                Return CopyFromScreen(_tl, _targetWindowInfo.rcClient.Width, _targetWindowInfo.rcClient.Height)
            End If
        End If
    End Function

A little debugging / testing...
Just take a screenshot of an app (in my case, pinball.exe because it's stock) 15 times, as fast as possible, and see how long it take. ie. how many scans can it do per second? hopefully at least 1! preferable 4-5 ? Anyway, I'm on an old pc (Pentium 4 cpu), so hopefully your results will be a little faster.

Public Sub RunFPSTest()

        If MessageBox.Show("Open pinball.exe before you click OK") = DialogResult.OK Then

            Dim _ss As Bitmap
            Dim _timeLapse As TimeSpan
            Dim _startTime As DateTime = Date.Now
            Dim _aColor As Color
            Dim _ctr As Int32 = 0

            For jj = 1 To 15
                _ss = GetWindowScreen("3D Pinball for Windows - Space Cadet", True)
                For xx As Int32 = 0 To _ss.Width - 1
                    For yy As Int32 = 0 To _ss.Height - 1
                        _aColor = _ss.GetPixel(xx, yy)
                        If _aColor.R = 0 And _aColor.G = 0 And _aColor.B = 0 Then
                            'white/black
                            _ctr += 1
                        End If
                    Next
                Next
            Next
            _timeLapse = Date.Now.Subtract(_startTime)
            MsgBox("It took " + _timeLapse.TotalSeconds.ToString & "s to scan 15 times, that's..." & Environment.NewLine _
                   & (15 / _timeLapse.TotalSeconds).ToString + " / second " & Environment.NewLine _
                   * "Image size: " & _ss.Width & "x" & _ss.Height & " pixels" & Environment.NewLine _
                   & "black_pixel_ctr: " & _ctr.ToString)
        End If

    End Sub

So, I must admit, 2.** fps (I mean, scans per second) seems really slow. In terms of game fps, it seriously is, but if you consider how big the image was, and actually how many pixels it had to scan over, it's quite a lot. Pinball was roughly 600,460 which is 2,83,608 pixels! That's 2 millions pixels it has to scan, every single time. My point, keep your scan areas small, and you can actually get really good scans-per-second. Doing the entire screen this way is NOT good.

oo..and the API/structdeclarations

Private Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As IntPtr
Private Declare Function GetWindowInfo Lib "user32" (ByVal hWnd As IntPtr, ByRef pWinInfo As WINDOWINFO) As Boolean

    Private Structure WINDOWINFO
        Dim cbSize As Integer
        Dim rcWindow As RECT_win32 ''entire form, including border
        Dim rcClient As RECT_win32 ''client window, no border
        Dim dwStyle As Integer 
        Dim dwExStyle As Integer
        Dim dwWindowStatus As UInt32 ''fullscreen? minimized?
        Dim cxWindowBorders As UInt32
        Dim cyWindowBorders As UInt32''not sure
        Dim atomWindowType As UInt16
        Dim wCreatorVersion As Short
    End Structure

''' <summary>
    ''' NOT bit compatible with system.drawing.rectangle
    ''' </summary>
    Private Structure RECT_win32 'not bit compatible with system.drawing.rectangle.
        Dim Left As Integer
        Dim Top As Integer
        Dim Right As Integer
        Dim Bottom As Integer
        Public Property Height() As Integer
            Get
                Return Bottom - Top
            End Get
            Set(ByVal value As Integer)
                Bottom = value - Top
            End Set
        End Property
        Public Property Width() As Integer
            Get
                Return Right - Left
            End Get
            Set(ByVal value As Integer)
                Right = value + Left
            End Set
        End Property
    End Structure

TODO: Look into using Marshal class for quicker (array) access to the bitmap's data


--my original account is abuckau907 , currently under 3 day ban.

Hope this helps.