vb.net screenshot tutorial

The Goal
Basically, take screenshots of a specific app (game?) and 'scan' (ie. analyze) the image for certain things (ie. colors: health bar, ememies, etc).

Basic Assumptions
You already know how to program in vb.net. You know variables, functions, classes, ByRef: most of the basics. I'm not going to say "Create new project" or specifically say where to declare variables.


Getting Started
(I lied, make a new project :D )

1) How exactly do we take the screenshot?
2) How do we scan over (analyze) that screenshot?
3) What is a screenshot?

..I'm answering in reverse order, but...

3) Since we are using Microsoft's .Net Framework for our project, we're going to use a few .net classes/functions. Microsoft likes 'bitmaps'(reference needed?), so if we want to leverage existing .net code, there are a few things we 'have' to do. Back to the point, our screenshot will be stored as a 'Bitmap' object. (probably System.Drawing.Bitmap? I don't have msvs open atm. google tells)
2) The bitmap class is pretty complex, and I'm not a graphics expert. The class has a function .GetPixel(x,y) which will return a 'color' object that we want. X,Y being the location in the bitmap of course. MAJOR point: It would be faster! to store the data in an array, and access it that way, instead of calling a .Function() for each and every pixel in the data! But..it works, and it's not much code. There will be a part 2 to this tutorial eventually. anyway.
1) The .net framework of course. There is a class called 'Graphics' and a method called 'CopyFromScreen'. <--Screen = your physical monitor, ie. what you're looking at ie. screenshot area

So, on to some code.
I created a class called "GooeyManager" (play on words), but feel free to add it wherever you want. All of these functions can/should be made static, ie. you don't need an object of the guimgr class before you can access the functions. Change it if you want to.

Just a demo

Code:

---

Dim _myBitmap as new Bitmap(50,50) '' 50,50 is the size: width, height, few diff. overloads
Dim _gr As Graphics = Graphics.FromImage(_myBitmap) '' we need the gr object for copyfromscreen

_gr.CopyFromScreen(***,***) '' not actual paremeters.
_gr.Dispose() ''don't forget about this later

---

_myBitmap : object of the Bitmap class. Stores the ss (screenshot) for us.
_gr : Graphics class. It's a little complex. CopyFromScreen() is pretty straight forward.
.CopyFromScreen() : ie. Take screenshot (of specific region on the monitor)

[some real code]

Code:

---

    Public Shared Function CopyFromScreen(ByVal tl As Point, ByVal width As Int32, ByVal height As Int32) As Bitmap
        Dim _rtnBmp As New Bitmap(width, height)
        Dim _gr As Graphics = Graphics.FromImage(_rtnBmp)
        _gr.CopyFromScreen(tl, New Point(0, 0), New Size(width, height), CopyPixelOperation.SourceCopy) '' sourcecopy or ____ ?
        _gr.Dispose()
        Return _rtnBmp
    End Function

---

description of parameters/arguments
tl : top-left, the top-left {x,y} of the region you want to copy
br : bottom-right, the bottom-right {x,y} of the region you want to copy

new Point(0,0) and New Size(*,*) : basically CopyFromScreen() doesn't have to use our entire bitmap.


That's it!
Kind of short, not a lot of code, but it can sometimes be useful.

Just a little more, but you might not need this.
take a ss of a specific form
Basically it users a few more Windows API to find a specific window (app,game,whatever) and take a screenshot of only that window (assuming it's open and top-most..)

Code:

---

    Public Shared Function GetWindowScreen(ByVal windowTitle As String, Optional ByVal includeBorders As Boolean = True) As Bitmap
        Dim _targetWindowHandle As IntPtr = FindWindow(Nothing, windowTitle) 'try to find window based on title.
        If _targetWindowHandle = 0 Then
            ''window not found
            MessageBox.Show("ScreenShotManager::GetScreenOfApp windowTitle not found! --> '" & windowTitle & "'" & Environment.NewLine _
                            & "Are you sure that program is running?")
            Return New Bitmap(2, 2) ''why not (1,1). choice.
        Else
            ''window found
            Dim _targetWindowInfo As WINDOWINFO
            GetWindowInfo(_targetWindowHandle, _targetWindowInfo)
            If includeBorders Then
                Dim _tl As New Point(_targetWindowInfo.rcWindow.Left, _targetWindowInfo.rcWindow.Top)
                Return CopyFromScreen(_tl, _targetWindowInfo.rcWindow.Width, _targetWindowInfo.rcWindow.Height)
            Else
                Dim _tl As New Point(_targetWindowInfo.rcClient.Left, _targetWindowInfo.rcClient.Top)
                Return CopyFromScreen(_tl, _targetWindowInfo.rcClient.Width, _targetWindowInfo.rcClient.Height)
            End If
        End If
    End Function

---

A little debugging / testing...
Just take a screenshot of an app (in my case, pinball.exe because it's stock) 15 times, as fast as possible, and see how long it take. ie. how many scans can it do per second? hopefully at least 1! preferable 4-5 ? Anyway, I'm on an old pc (Pentium 4 cpu), so hopefully your results will be a little faster.

Code:

---

Public Sub RunFPSTest()

        If MessageBox.Show("Open pinball.exe before you click OK") = DialogResult.OK Then

            Dim _ss As Bitmap
            Dim _timeLapse As TimeSpan
            Dim _startTime As DateTime = Date.Now
            Dim _aColor As Color
            Dim _ctr As Int32 = 0

            For jj = 1 To 15
                _ss = GetWindowScreen("3D Pinball for Windows - Space Cadet", True)
                For xx As Int32 = 0 To _ss.Width - 1
                    For yy As Int32 = 0 To _ss.Height - 1
                        _aColor = _ss.GetPixel(xx, yy)
                        If _aColor.R = 0 And _aColor.G = 0 And _aColor.B = 0 Then
                            'white/black
                            _ctr += 1
                        End If
                    Next
                Next
            Next
            _timeLapse = Date.Now.Subtract(_startTime)
            MsgBox("It took " + _timeLapse.TotalSeconds.ToString & "s to scan 15 times, that's..." & Environment.NewLine _
                  & (15 / _timeLapse.TotalSeconds).ToString + " / second " & Environment.NewLine _
                  * "Image size: " & _ss.Width & "x" & _ss.Height & " pixels" & Environment.NewLine _
                  & "black_pixel_ctr: " & _ctr.ToString)
        End If

    End Sub

---

https://i48.tinypic.com/1fwdv4.jpg

So, I must admit, 2.** fps (I mean, scans per second) seems really slow. In terms of game fps, it seriously is, but if you consider how big the image was, and actually how many pixels it had to scan over, it's quite a lot. Pinball was roughly 600,460 which is 2,83,608 pixels! That's 2 millions pixels it has to scan, every single time. My point, keep your scan areas small, and you can actually get really good scans-per-second. Doing the entire screen this way is NOT good.

oo..and the API/structdeclarations

Code:

---

Private Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As IntPtr
Private Declare Function GetWindowInfo Lib "user32" (ByVal hWnd As IntPtr, ByRef pWinInfo As WINDOWINFO) As Boolean

    Private Structure WINDOWINFO
        Dim cbSize As Integer
        Dim rcWindow As RECT_win32 ''entire form, including border
        Dim rcClient As RECT_win32 ''client window, no border
        Dim dwStyle As Integer
        Dim dwExStyle As Integer
        Dim dwWindowStatus As UInt32 ''fullscreen? minimized?
        Dim cxWindowBorders As UInt32
        Dim cyWindowBorders As UInt32''not sure
        Dim atomWindowType As UInt16
        Dim wCreatorVersion As Short
    End Structure

''' <summary>
    ''' NOT bit compatible with system.drawing.rectangle
    ''' </summary>
    Private Structure RECT_win32 'not bit compatible with system.drawing.rectangle.
        Dim Left As Integer
        Dim Top As Integer
        Dim Right As Integer
        Dim Bottom As Integer
        Public Property Height() As Integer
            Get
                Return Bottom - Top
            End Get
            Set(ByVal value As Integer)
                Bottom = value - Top
            End Set
        End Property
        Public Property Width() As Integer
            Get
                Return Right - Left
            End Get
            Set(ByVal value As Integer)
                Right = value + Left
            End Set
        End Property
    End Structure

---

TODO: Look into using Marshal class for quicker (array) access to the bitmap's data


--my original account is abuckau907 , currently under 3 day ban.

Hope this helps.

Another way of doing it using gdi32.dll and user32.dll API's

Code:

---

Public Class User32
        <StructLayout(LayoutKind.Sequential)> _
        Public Structure RECT
            Public left As Integer
            Public top As Integer
            Public right As Integer
            Public bottom As Integer
        End Structure


        Declare Function GetDesktopWindow Lib "user32.dll" () As IntPtr
        Declare Function GetWindowDC Lib "user32.dll" (ByVal hwnd As IntPtr) As IntPtr
        Declare Function ReleaseDC Lib "user32.dll" (ByVal hwnd As IntPtr, ByVal hdc As IntPtr) As Int32
        Declare Function GetWindowRect Lib "user32.dll" (ByVal hwnd As IntPtr, ByRef lpRect As RECT) As Int32

Private Class GDI32
        Public SRCCOPY As Integer = &HCC0020
        Declare Function BitBlt Lib "gdi32.dll" ( _
            ByVal hDestDC As IntPtr, _
            ByVal x As Int32, _
            ByVal y As Int32, _
            ByVal nWidth As Int32, _
            ByVal nHeight As Int32, _
            ByVal hSrcDC As IntPtr, _
            ByVal xSrc As Int32, _
            ByVal ySrc As Int32, _
            ByVal dwRop As Int32) As Int32


        Declare Function CreateCompatibleBitmap Lib "gdi32.dll" (ByVal hdc As IntPtr, ByVal nWidth As Int32, ByVal nHeight As Int32) As IntPtr
        Declare Function CreateCompatibleDC Lib "gdi32.dll" (ByVal hdc As IntPtr) As IntPtr
        Declare Function DeleteDC Lib "gdi32.dll" (ByVal hdc As IntPtr) As Int32
        Declare Function DeleteObject Lib "gdi32.dll" (ByVal hObject As IntPtr) As Int32
        Declare Function SelectObject Lib "gdi32.dll" (ByVal hdc As IntPtr, ByVal hObject As IntPtr) As IntPtr
    End Class

Public Shared Function CaptureDeskTopRectangle(ByVal CapRect As Rectangle, ByVal CapRectWidth As Integer, ByVal CapRectHeight As Integer) As Bitmap
        Dim SC As New ScreenCap
        Dim bmpImage As New Bitmap(SC.CaptureScreen)
        Dim bmpCrop As New Bitmap(CapRectWidth, CapRectHeight, bmpImage.PixelFormat)
        Dim recCrop As New Rectangle(CapRect.X, CapRect.Y, CapRectWidth, CapRectHeight)
        Dim gphCrop As Graphics = Graphics.FromImage(bmpCrop)
        Dim recDest As New Rectangle(0, 0, CapRectWidth, CapRectHeight)
        gphCrop.DrawImage(bmpImage, recDest, recCrop.X, recCrop.Y, recCrop.Width, recCrop.Height, GraphicsUnit.Pixel)
        Return bmpCrop
    End Function

Public Shared Function CaptureScreen() As Image
        Return CaptureWindow(User32.GetDesktopWindow())
    End Function


    Public Shared Function CaptureWindow(ByVal handle As IntPtr) As Image
        Dim SRCCOPY As Integer = &HCC0020
        Dim hdcSrc As IntPtr = User32.GetWindowDC(handle)
        Dim windowRect As New User32.RECT
        User32.GetWindowRect(handle, windowRect)
        Dim width As Integer = windowRect.right - windowRect.left
        Dim height As Integer = windowRect.bottom - windowRect.top
        Dim hdcDest As IntPtr = GDI32.CreateCompatibleDC(hdcSrc)
        Dim hBitmap As IntPtr = GDI32.CreateCompatibleBitmap(hdcSrc, width, height)
        Dim hOld As IntPtr = GDI32.SelectObject(hdcDest, hBitmap)
        GDI32.BitBlt(hdcDest, 0, 0, width, height, hdcSrc, 0, 0, SRCCOPY)
        GDI32.SelectObject(hdcDest, hOld)
        GDI32.DeleteDC(hdcDest)
        User32.ReleaseDC(handle, hdcSrc)
        Dim img As Image = Image.FromHbitmap(hBitmap)
        GDI32.DeleteObject(hBitmap)
        Return img
    End Function

---

This is from my capture image capture class

and @abuckau907_num2
You never included the API's that you've used. Such as FindWindow

(oops: edited in the other api / struct)
I don't know much about how windows internally handles the graphics. I keep seeing dc, device context? display context? I'll have to read up on it soon. Would be nice to get a little more performance out of mine, but it serves its purpose. Thnx for posting your code.

I went to see the hobbit in 3D :D I should have finished editing original tut before posting, thought it was just about finished. I posted 1/2 ( 1 of 2) of the api's used :p

Another little piece, with different API, to get only 1 pixel.

Code:

---

    Private Declare Function GetDesktopWindow Lib "user32" () As IntPtr
    Private Declare Function GetWindowDC Lib "user32" (ByVal hWnd As IntPtr) As IntPtr
    Private Declare Function GetPixel Lib "gdi32" (ByVal hdc As IntPtr, ByVal XPos As Integer, YPos As Integer) As UInteger
    Private Declare Function ReleaseDC Lib "user32" (ByVal hWnd As IntPtr, ByVal hDC As IntPtr) As Boolean

Public Shared Function GetPixelColor(ByVal x As Int32, ByVal y As Int32) As Color
        Dim hWnd As IntPtr
        Dim hDC As IntPtr
        hWnd = GetDesktopWindow()
        hDC = GetWindowDC(hWnd)

        Dim lColor As Int32 = GetPixel(hDC, x, y)
        ReleaseDC(hWnd, hDC)

        Return ColorTranslator.FromWin32(lColor)
    End Function

---

^^might be useful if know you the exact location of a pixel and that location won't change.

Reference the functions from winapi to MSDN.. So people can actually learn and have a proper explanation on what it does.. ._.